Cloud Firestore – From JSON Deserialization to Object Document Mapping (ODM)Minh Dao
This talk deals with a number of techniques that help you get data from Firestore into your Flutter application in a structured and safe manner.
- Updated Firebase setup
- Conventional json parsing
- Enhanced enum classes
- deep_pick library
- json_serializable library
- withConverter
- Firestore ODM
Nodester is an open source platform that provides free Node.js hosting. It currently hosts around 2,000 apps and is built on Amazon EC2 instances. The main components are proxy.js which handles all HTTP and Websocket requests, and app.js which handles the REST API, CLI, and nodester.com site. The roadmap includes separating out features, reorganizing the system layout, looking at alternative sandboxing, and potentially breaking components out to their own VMs.
Nodester is an open source platform that provides free Node.js hosting. It currently hosts around 2,000 apps and is built on Amazon EC2 instances. The main components are proxy.js which handles all HTTP and Websocket requests, and app.js which handles the REST API, CLI, and nodester.com site. The system utilizes chroot and Unionfs for sandboxing apps, and the roadmap includes separating out features, reorganizing the system layout, and potentially exploring alternative sandboxing methods like SELinux.
Spatial MongoDB, Node.JS, and Express - server-side JS for your applicationSteven Pousty
This document summarizes a presentation about building spatial web services using MongoDB and Node.js. The presentation covers loading spatial data into MongoDB, creating 2D spatial indexes, performing spatial queries, and building web services to access the spatial data. It is aimed at developers who already know MongoDB and Node.js, and assumes basic familiarity with the MongoDB command line. The live demo shows examples of spatial queries and operations on sample data.
This document discusses Kafka and message buses. It describes synchronous vs asynchronous communication and how a message bus can be used for event aggregation, CQRS, microservices, and event sourcing. It then provides details on Apache Kafka, including its creation by LinkedIn, use by major companies, and basic workflow. Key components of the Kafka stack like Zookeeper, brokers, producers, consumers and topics are explained. The document ends with an example of launching a local Kafka cluster and producing and consuming messages.
Monitoring a Kubernetes-backed microservice architecture with PrometheusFabian Reinartz
As many startups of the last decade, SoundCloud’s architecture started as a Ruby-on-Rails monolith, which later had to be broken into microservices to cope with the growing size and complexity of the site. The microservices initially ran on an in-house container management and deployment platform. Recently, the company has started to migrate to Kubernetes.
With the introduction of microservices, the existing conventional monitoring setup failed both conceptually and in terms of scalability. Thus, starting in 2012, SoundCloud invested heavily into the development of the open-source monitoring system Prometheus, which was designed for large-scale highly dynamic service-oriented architectures.
Migrating to Kubernetes, it became apparent that Prometheus and Kubernetes are a match made in open-source heaven. The talk will demonstrate the current Prometheus setup at SoundCloud, monitoring a large-scale Kubernetes cluster.
This document discusses service discovery options in Prometheus. It describes setting up infrastructure with Prometheus, Grafana, Consul and Spring Boot applications. It then covers different service discovery mechanisms like static configuration, DNS, and Consul. DNS and Consul configurations are demonstrated, with Consul used to register microservices that Prometheus then discovers and scrapes. The document aims to showcase service discovery capabilities and options in Prometheus.
Cloud Firestore – From JSON Deserialization to Object Document Mapping (ODM)Minh Dao
This talk deals with a number of techniques that help you get data from Firestore into your Flutter application in a structured and safe manner.
- Updated Firebase setup
- Conventional json parsing
- Enhanced enum classes
- deep_pick library
- json_serializable library
- withConverter
- Firestore ODM
Nodester is an open source platform that provides free Node.js hosting. It currently hosts around 2,000 apps and is built on Amazon EC2 instances. The main components are proxy.js which handles all HTTP and Websocket requests, and app.js which handles the REST API, CLI, and nodester.com site. The roadmap includes separating out features, reorganizing the system layout, looking at alternative sandboxing, and potentially breaking components out to their own VMs.
Nodester is an open source platform that provides free Node.js hosting. It currently hosts around 2,000 apps and is built on Amazon EC2 instances. The main components are proxy.js which handles all HTTP and Websocket requests, and app.js which handles the REST API, CLI, and nodester.com site. The system utilizes chroot and Unionfs for sandboxing apps, and the roadmap includes separating out features, reorganizing the system layout, and potentially exploring alternative sandboxing methods like SELinux.
Spatial MongoDB, Node.JS, and Express - server-side JS for your applicationSteven Pousty
This document summarizes a presentation about building spatial web services using MongoDB and Node.js. The presentation covers loading spatial data into MongoDB, creating 2D spatial indexes, performing spatial queries, and building web services to access the spatial data. It is aimed at developers who already know MongoDB and Node.js, and assumes basic familiarity with the MongoDB command line. The live demo shows examples of spatial queries and operations on sample data.
This document discusses Kafka and message buses. It describes synchronous vs asynchronous communication and how a message bus can be used for event aggregation, CQRS, microservices, and event sourcing. It then provides details on Apache Kafka, including its creation by LinkedIn, use by major companies, and basic workflow. Key components of the Kafka stack like Zookeeper, brokers, producers, consumers and topics are explained. The document ends with an example of launching a local Kafka cluster and producing and consuming messages.
Monitoring a Kubernetes-backed microservice architecture with PrometheusFabian Reinartz
As many startups of the last decade, SoundCloud’s architecture started as a Ruby-on-Rails monolith, which later had to be broken into microservices to cope with the growing size and complexity of the site. The microservices initially ran on an in-house container management and deployment platform. Recently, the company has started to migrate to Kubernetes.
With the introduction of microservices, the existing conventional monitoring setup failed both conceptually and in terms of scalability. Thus, starting in 2012, SoundCloud invested heavily into the development of the open-source monitoring system Prometheus, which was designed for large-scale highly dynamic service-oriented architectures.
Migrating to Kubernetes, it became apparent that Prometheus and Kubernetes are a match made in open-source heaven. The talk will demonstrate the current Prometheus setup at SoundCloud, monitoring a large-scale Kubernetes cluster.
This document discusses service discovery options in Prometheus. It describes setting up infrastructure with Prometheus, Grafana, Consul and Spring Boot applications. It then covers different service discovery mechanisms like static configuration, DNS, and Consul. DNS and Consul configurations are demonstrated, with Consul used to register microservices that Prometheus then discovers and scrapes. The document aims to showcase service discovery capabilities and options in Prometheus.
Collect distributed application logging using fluentd (EFK stack)Marco Pas
This document discusses using Fluentd to collect distributed application logging in a containerized environment. It provides an overview of Fluentd, including its pluggable architecture and configuration. It then demonstrates capturing logging from Docker containers and HTTP services and storing the logs in MongoDB and Elasticsearch as part of the ELK stack. It shows filtering and parsing logs from a Spring Boot application. Finally, it discusses setting up Fluentd for high availability.
The document discusses the benefits and challenges of proof-driven development using the Coq proof assistant. It describes how Coq can be used to formally prove properties about code during development. However, it also notes that Coq has limitations when dealing with large numbers, which can cause stack overflows. It also discusses using Coq to formally specify the MessagePack serialization format and prove properties about it.
Chronix as Long-Term Storage for PrometheusQAware GmbH
Cloud Native Con 2016, Seattle: Talk by Moritz Kammerer (@phxql, Senior Software Engineer at QAware).
Abstract: Prometheus is great when it comes to monitoring and alerting. But the long term storage opportunities are comparatively weak compared to related time series databases (missing data distribution, sharding etc.). At this point Chronix enters the stage. Chronix is an open source time series database. It focuses on an efficient long term storage both in terms of storage volume and access times. Chronix achieves a compression rate of 98% compared to data in CSV files while an average query took 21 milliseconds, determined in a benchmark asking 96 queries for different time ranges and time series. Chronix offers a multi-dimensional generic data model for storing all kinds of time series, functions for anomaly detection used in the frameworks EGADS and SAX, and an integration with Apache Spark allows for distributed time series processing. In this code-intense session we show the integration of Prometheus and Chronix. We also dig into the details of Chronix and explain why Chronix <3 Prometheus and vice versa. Furthermore we demonstrate a toolchain: collect data with Prometheus, pipe them to Chronix, visualize both data sources in Grafana [5], and easily analyze tons of data with Spark and Apache Zeppelin.
The document discusses reactive programming in Swift. It presents three cases of using reactive programming: 1) using an Observable with an array and UITableView, 2) using a UISearchBar with URL requests and a UITableView, and 3) using MVVM architecture. Live coding demonstrations are provided on GitHub for the first two cases. Useful links on RxSwift are also provided, and contact information is given for the presenter.
The document summarizes the efforts of three individuals to break the RPiDocker challenge of running the most Docker containers on a Raspberry Pi 2. They took a methodical approach, measuring performance and automating setup. Key steps included systemd and Docker tuning, using a highly optimized web server, and addressing the RPi2's network namespace limit. Collaboration and sharing ideas helped optimize the process and ultimately run 2499 containers, hitting a Go runtime thread limit. Working around the limit confirmed the ability to run nearly 2740 containers before hitting memory limits.
Titus AWS VPC networking for containersAndrew Leung
The document describes how Titus, a Mesos container orchestrator, implements networking for tasks and containers running on Amazon EC2 instances. It discusses how Titus integrates with the Docker engine to configure container networking, allocates IPs and security groups, and sets up Linux routing and traffic control rules to connect containers to their respective virtual networks while isolating network traffic between tasks. Special routing tables and rules are used to implement network isolation and connectivity between containers, EC2 network interfaces, and the host.
Improving monitoring systems Interoperability with OpenMetricsChan Shik Lim
Introduction Prometheus with OpenMetrics for improving monitoring system interoperability.
- Introduction Prometheus
- Introduction OpenMetrics
- Demo: Prometheus and Datadog integration with OpenMetrics
This document discusses the C100K problem of handling 100,000 concurrent network connections efficiently and describes how the Go programming language solves this problem. It explains that Go uses lightweight goroutines instead of OS threads, has a fast scheduler, and uses non-blocking I/O with epoll to efficiently handle a large number of clients with a small memory footprint on each CPU core. An example TCP/HTTP server is shown to demonstrate how Go implements networking.
This document discusses strategies for scaling Prometheus monitoring to millions of machines. It describes the stages of scaling from manual Prometheus configuration to per-region Prometheus instances. Issues with input/output and retention led to further scaling techniques like sharding, using Kafka as a message bus, downsampling metrics, and developing an agent and query API. The talk introduces "Vulcan" which implements Prometheus APIs with Cassandra storage and Kafka input to enable monitoring at large scale.
This document discusses M3, Uber's time series database. It provides an overview of M3 and compares it to Graphite, which Uber previously used. M3 was built to have better resiliency, efficiency, and scalability than Graphite. It provides both a Graphite-compatible query interface and its own query language called M3QL. The document describes M3's architecture, storage, indexing, and how it handles high write and read throughput. It also covers instrumentation, profiling, load testing, and optimizations used in M3's Go code.
How bol.com makes sense of its logs, using the Elastic technology stack.Renzo Tomà
Bol.com uses the Elastic (ELK) stack to make sense of logs from over 1,600 servers and 500-600 million events per day. Key aspects of their system include:
1. Shipping JSON-formatted log events from sources like Apache, databases, and applications to Redis queues to allow multiple Logstash instances to process events in real-time without data loss.
2. Enriching log events with information like request IDs to correlate requests across services, and IP-to-role mappings to identify client roles.
3. Using Elasticsearch aggregations and transformations to generate a directed graph of service dependencies based on logs, to help understand their distributed architecture.
Payment channels allow for off-chain transactions between two parties to enable trustless, high-volume, low-cost micropayments on the Bitcoin blockchain. The key aspects are: (1) a multi-signature wallet is set up and funded, (2) off-chain balance updates occur via time-locked transactions, (3) the final on-chain transaction closes the channel by broadcasting the final signed state. Hashed Timelock Contracts (HTLCs) extend this to enable payments across multiple channels by using hash preimages and timelocks to guarantee delivery or refunds. Bidirectional channels allow payments in both directions through commitment transactions signed by both parties.
This document discusses simulating switch traffic in a virtualized OpenFlow testbed using the FlowSim simulator. It provides details on experiments conducted in a related paper that analyzed the performance of OpenFlow controllers and the impact of delay on ICMP, TCP, and UDP traffic. The simulation will focus on simulating the traffic through switch S2.
Blockchain for creative content - What we do in LikeCoinAludirk Wong
The document discusses the LikeCoin project, which uses blockchain technology to create a decentralized ecosystem for creative content. It aims to address technical problems with Ethereum such as slow transaction speeds and high fees. LikeCoin will use Ethereum initially and later transition to its own sidechain called LikeChain based on Tendermint for higher throughput and lower fees. It will also utilize IPFS and IPLD to store content and metadata in a decentralized manner. The document provides examples of open source projects like puttyimages that are part of building out the LikeCoin ecosystem.
Herding cats & catching fire: Workday's telemetry & middlewareSensu Inc.
In this talk from Sensu Summit 2018, David Beaurpere, Principal Software Engineer for the Observability Group at Workday Ltd, discusses how Sensu 1.x evolved from a Nagios replacement to the backbone monitoring data collection and transport at Workday.
SignalR: Add real-time to your applicationsEugene Zharkov
SignalR allows adding real-time functionality to applications by using multiple transport methods for client-server communication like WebSockets, server-sent events, and long polling. It supports clients on various platforms through libraries like jQuery, Mono, and QT. The core abstractions in SignalR are the PersistentConnection for raw connections and the Hub for a higher-level API. Applications can create Hub objects on the server to define methods for clients to call, and subscribe to events from the client side in various languages like C#, JavaScript, and C#.
This document provides an introduction to Ethereum and how it works at a high level. It discusses Ethereum's state transition function and how it updates the world state based on transactions. It describes Ethereum clients like Geth and Parity that allow users to interact with the Ethereum network. It also summarizes how accounts work in Ethereum, how transactions are processed, how contracts are deployed and executed, and how mining secures the network through proof-of-work.
Collect distributed application logging using fluentd (EFK stack)Marco Pas
This document discusses using Fluentd to collect distributed application logging in a containerized environment. It provides an overview of Fluentd, including its pluggable architecture and configuration. It then demonstrates capturing logging from Docker containers and HTTP services and storing the logs in MongoDB and Elasticsearch as part of the ELK stack. It shows filtering and parsing logs from a Spring Boot application. Finally, it discusses setting up Fluentd for high availability.
The document discusses the benefits and challenges of proof-driven development using the Coq proof assistant. It describes how Coq can be used to formally prove properties about code during development. However, it also notes that Coq has limitations when dealing with large numbers, which can cause stack overflows. It also discusses using Coq to formally specify the MessagePack serialization format and prove properties about it.
Chronix as Long-Term Storage for PrometheusQAware GmbH
Cloud Native Con 2016, Seattle: Talk by Moritz Kammerer (@phxql, Senior Software Engineer at QAware).
Abstract: Prometheus is great when it comes to monitoring and alerting. But the long term storage opportunities are comparatively weak compared to related time series databases (missing data distribution, sharding etc.). At this point Chronix enters the stage. Chronix is an open source time series database. It focuses on an efficient long term storage both in terms of storage volume and access times. Chronix achieves a compression rate of 98% compared to data in CSV files while an average query took 21 milliseconds, determined in a benchmark asking 96 queries for different time ranges and time series. Chronix offers a multi-dimensional generic data model for storing all kinds of time series, functions for anomaly detection used in the frameworks EGADS and SAX, and an integration with Apache Spark allows for distributed time series processing. In this code-intense session we show the integration of Prometheus and Chronix. We also dig into the details of Chronix and explain why Chronix <3 Prometheus and vice versa. Furthermore we demonstrate a toolchain: collect data with Prometheus, pipe them to Chronix, visualize both data sources in Grafana [5], and easily analyze tons of data with Spark and Apache Zeppelin.
The document discusses reactive programming in Swift. It presents three cases of using reactive programming: 1) using an Observable with an array and UITableView, 2) using a UISearchBar with URL requests and a UITableView, and 3) using MVVM architecture. Live coding demonstrations are provided on GitHub for the first two cases. Useful links on RxSwift are also provided, and contact information is given for the presenter.
The document summarizes the efforts of three individuals to break the RPiDocker challenge of running the most Docker containers on a Raspberry Pi 2. They took a methodical approach, measuring performance and automating setup. Key steps included systemd and Docker tuning, using a highly optimized web server, and addressing the RPi2's network namespace limit. Collaboration and sharing ideas helped optimize the process and ultimately run 2499 containers, hitting a Go runtime thread limit. Working around the limit confirmed the ability to run nearly 2740 containers before hitting memory limits.
Titus AWS VPC networking for containersAndrew Leung
The document describes how Titus, a Mesos container orchestrator, implements networking for tasks and containers running on Amazon EC2 instances. It discusses how Titus integrates with the Docker engine to configure container networking, allocates IPs and security groups, and sets up Linux routing and traffic control rules to connect containers to their respective virtual networks while isolating network traffic between tasks. Special routing tables and rules are used to implement network isolation and connectivity between containers, EC2 network interfaces, and the host.
Improving monitoring systems Interoperability with OpenMetricsChan Shik Lim
Introduction Prometheus with OpenMetrics for improving monitoring system interoperability.
- Introduction Prometheus
- Introduction OpenMetrics
- Demo: Prometheus and Datadog integration with OpenMetrics
This document discusses the C100K problem of handling 100,000 concurrent network connections efficiently and describes how the Go programming language solves this problem. It explains that Go uses lightweight goroutines instead of OS threads, has a fast scheduler, and uses non-blocking I/O with epoll to efficiently handle a large number of clients with a small memory footprint on each CPU core. An example TCP/HTTP server is shown to demonstrate how Go implements networking.
This document discusses strategies for scaling Prometheus monitoring to millions of machines. It describes the stages of scaling from manual Prometheus configuration to per-region Prometheus instances. Issues with input/output and retention led to further scaling techniques like sharding, using Kafka as a message bus, downsampling metrics, and developing an agent and query API. The talk introduces "Vulcan" which implements Prometheus APIs with Cassandra storage and Kafka input to enable monitoring at large scale.
This document discusses M3, Uber's time series database. It provides an overview of M3 and compares it to Graphite, which Uber previously used. M3 was built to have better resiliency, efficiency, and scalability than Graphite. It provides both a Graphite-compatible query interface and its own query language called M3QL. The document describes M3's architecture, storage, indexing, and how it handles high write and read throughput. It also covers instrumentation, profiling, load testing, and optimizations used in M3's Go code.
How bol.com makes sense of its logs, using the Elastic technology stack.Renzo Tomà
Bol.com uses the Elastic (ELK) stack to make sense of logs from over 1,600 servers and 500-600 million events per day. Key aspects of their system include:
1. Shipping JSON-formatted log events from sources like Apache, databases, and applications to Redis queues to allow multiple Logstash instances to process events in real-time without data loss.
2. Enriching log events with information like request IDs to correlate requests across services, and IP-to-role mappings to identify client roles.
3. Using Elasticsearch aggregations and transformations to generate a directed graph of service dependencies based on logs, to help understand their distributed architecture.
Payment channels allow for off-chain transactions between two parties to enable trustless, high-volume, low-cost micropayments on the Bitcoin blockchain. The key aspects are: (1) a multi-signature wallet is set up and funded, (2) off-chain balance updates occur via time-locked transactions, (3) the final on-chain transaction closes the channel by broadcasting the final signed state. Hashed Timelock Contracts (HTLCs) extend this to enable payments across multiple channels by using hash preimages and timelocks to guarantee delivery or refunds. Bidirectional channels allow payments in both directions through commitment transactions signed by both parties.
This document discusses simulating switch traffic in a virtualized OpenFlow testbed using the FlowSim simulator. It provides details on experiments conducted in a related paper that analyzed the performance of OpenFlow controllers and the impact of delay on ICMP, TCP, and UDP traffic. The simulation will focus on simulating the traffic through switch S2.
Blockchain for creative content - What we do in LikeCoinAludirk Wong
The document discusses the LikeCoin project, which uses blockchain technology to create a decentralized ecosystem for creative content. It aims to address technical problems with Ethereum such as slow transaction speeds and high fees. LikeCoin will use Ethereum initially and later transition to its own sidechain called LikeChain based on Tendermint for higher throughput and lower fees. It will also utilize IPFS and IPLD to store content and metadata in a decentralized manner. The document provides examples of open source projects like puttyimages that are part of building out the LikeCoin ecosystem.
Herding cats & catching fire: Workday's telemetry & middlewareSensu Inc.
In this talk from Sensu Summit 2018, David Beaurpere, Principal Software Engineer for the Observability Group at Workday Ltd, discusses how Sensu 1.x evolved from a Nagios replacement to the backbone monitoring data collection and transport at Workday.
SignalR: Add real-time to your applicationsEugene Zharkov
SignalR allows adding real-time functionality to applications by using multiple transport methods for client-server communication like WebSockets, server-sent events, and long polling. It supports clients on various platforms through libraries like jQuery, Mono, and QT. The core abstractions in SignalR are the PersistentConnection for raw connections and the Hub for a higher-level API. Applications can create Hub objects on the server to define methods for clients to call, and subscribe to events from the client side in various languages like C#, JavaScript, and C#.
This document provides an introduction to Ethereum and how it works at a high level. It discusses Ethereum's state transition function and how it updates the world state based on transactions. It describes Ethereum clients like Geth and Parity that allow users to interact with the Ethereum network. It also summarizes how accounts work in Ethereum, how transactions are processed, how contracts are deployed and executed, and how mining secures the network through proof-of-work.
gRPC is a modern high performance RPC (Remote Procedure Call) framework that can run in any environment. gRPC is based on HTTP/2, Protocol Buffers and other modern standard-based technologies. Together with excellent tooling, it helps you create high throughput, low latency, real-time services.
The HTML5 WebSocket API allows for true full-duplex communication between a client and server. It uses the WebSocket protocol which provides a standardized way for the client to "upgrade" an HTTP connection to a WebSocket connection, allowing for messages to be sent in either direction at any time with very little overhead. This enables real-time applications that were previously difficult to achieve with traditional HTTP requests. Common server implementations include Kaazing WebSocket Gateway, Jetty, and Node.js. The JavaScript API provides an easy way for clients to connect, send, and receive messages via a WebSocket connection.
This document provides information about upgrading to ASP.NET Core 3.0, Blazor, gRPC, and SignalR. It includes links to documentation and tutorials about these technologies. Key topics covered include how Blazor works, using Blazor on the client or server, building a pizza store UI with Blazor, gRPC concepts like protocol buffers and remote procedure calls, demonstrating gRPC, advantages of gRPC like performance and multiple languages, and an overview of SignalR.
containerd is an industry-standard container runtime with an emphasis on simplicity, robustness and portability. It is available as a daemon for Linux and Windows, which can manage the complete container lifecycle of its host system: image transfer and storage, container execution and supervision, low-level storage and network attachments, etc.
containerd is designed to be embedded into a larger system, rather than being used directly by developers or end-users.
The document discusses several HTML5 APIs including Geolocation, Web Storage, Web SQL, Web Workers, Web Sockets, and Canvas. It provides code examples and explanations for how each API works and can be used, including getting a user's location, storing data locally, running scripts in parallel threads, enabling real-time communications, and drawing graphics. It also mentions some use cases for these technologies like building games, performing intensive calculations, and syntax highlighting.
Andrew Betts Web Developer, The Financial Times at Fastly Altitude 2016
Running custom code at the Edge using a standard language is one of the biggest advantages of working with Fastly’s CDN. Andrew gives you a tour of all the problems the Financial Times and Nikkei solve in VCL and how their solutions work.
QTP can be used to test non-GUI applications by accessing APIs and COM objects directly. This allows testing applications via Telnet, FTP, web services, and XML without using the GUI. It increases test speed and avoids synchronization issues. The document discusses using COM objects like WinHTTP and W3Sockets for Telnet, FTP, and web services. It also discusses using Msxml2.DOMDocument to work with XML files and compare them using QTP objects.
Cotopaxi - IoT testing toolkit (Black Hat Asia 2019 Arsenal)Jakub Botwicz
Presentation about Cotopaxi toolkit from Black Hat Asia 2019 Arsenal session. Author: Jakub Botwicz
https://www.blackhat.com/asia-19/arsenal/schedule/index.html#cotopaxi-iot-protocols-security-testing-toolkit-14325
DEVNET-1136 Cisco ONE Enterprise Cloud Suite for Infrastructure Management.Cisco DevNet
Traditional infrastructure management and administration has been about element managers, and box by box administration. Tools like Cisco ONE Enterprise Cloud Suite offer a platform for automation, but they are most often used through the respective web interfaces for building, executing, and checking status of tasks. Today it is becoming more important to be able to use APIs to programmatically access infrastructure as well as automation tools to enable customization and linking through tools like Python or native REST calls. This talk will focus on covering the basics of leveraging the APIs for the software in the Suite, as well as show some example use cases. Most of the discussion will be on UCS Director and Intercloud Fabric Director’s APIs.
Developers choose HTTP for its ubiquity. HTTP's semantics are cherry-picked or embraced in the myriad of apis we develop and consume. Efficiency discussions are commonplace: Does this design imply N+1 requests? Should we denormalize the model? How do consumers discover changes in state? How many connections are needed to effectively use this api?
Meanwhile, HTTP 1.1 is a choice, as opposed to constant. SPDY and HTTP/2 implementations surface, simultaneously retaining semantics and dramatically changing performance implications. We can choose treat these new protocols as more efficient versions HTTP 1.1 or buy into new patterns such as server-side push.
This session walks you through these topics via an open source project from Square called okhttp. You'll understand how okhttp addresses portability so that you can develop against something as familiar as java's HTTPUrlConnection. We'll review how to use new protocol features and constraints to keep in mind along the way. You'll learn how to sandbox ideas with okhttp's mock server, so that you can begin experimenting with SPDY and HTTP/2 today!
Polling Techniques, Ajax, protocol Switching from Http to Websocket standard ...Srikanth Reddy Pallerla
This document discusses WebSockets and how they provide a full-duplex communication channel over a single TCP connection that avoids some of the issues with traditional HTTP requests. WebSockets allow for real-time data transfer from server to client without polling. They utilize an initial HTTP request for handshake that upgrades the connection and then transmits message-based data similar to UDP with reliability of TCP. This reduces resource usage compared to techniques like long polling and streaming.
CocoaConf: The Language of Mobile Software is APIsTim Burks
We’re all excited about using the same language to write our mobile apps and cloud services, but as we do, we’ll still need to work with a few things that aren’t written with Swift. Fortunately, there are some great patterns that we can use for doing that. In this session we’ll talk about two technologies that you can use to make your app speak with APIs written in any language: OpenAPI and Protocol Buffers, and then we’ll see how to use them from clients and servers that are written in Swift.
Presented Friday November 4, 2016 in San Jose.
Hybrid Apps (Native + Web) via QtWebKitAriya Hidayat
This document discusses hybrid apps that combine native and web technologies using QtWebKit. It provides an overview of QtWebKit and how it can be used to embed web content in native apps. It describes various tools, technologies and frameworks that can be used to build hybrid apps, including JavaScript, CSS, Canvas, WebGL, and tools for debugging, testing, and designing hybrid apps. It concludes that web technologies are advancing rapidly and hybrid approaches can help migration, while tools still need to catch up to support building high quality hybrid apps.
Using Location Data to Showcase Keys, Windows, and Joins in Kafka Streams DSL...confluent
This document provides an overview of using location data to showcase keys, windows, and joins in Kafka Streams DSL and KSQL. It describes several algorithms for finding the nearest airport or aircraft to a given location within a 5-minute window using Kafka Streams. The algorithms involve bucketing location data, calculating distances, aggregating counts, and suppressing updates to optimize processing. The document cautions that windows are not magic and getting the window logic wrong can lead to incorrect results.
The document discusses new networking features in Go 1.8, including improvements to the net/http package. Some key points include:
- The net/http package now supports HTTP/2 server push and includes mechanisms for sending pushes from handlers.
- Context is emphasized as being critical, with additions of context support to packages like database/sql. All Lookup methods on net.Resolver now take a context.
- The crypto/tls package includes new features like Conn.CloseWrite and Config.Clone.
- Improvements to HTTP servers include graceful shutdown support via Server.Shutdown and configuration options like ReadHeaderTimeout.
- The expvar package provides an interface for exposing operational counters via HTTP
Backend, app e internet das coisas com NodeJS no Google Cloud PlatformDevMT
Talk apresentado no Cloud Next '17 Extended Cuiabá por Alvaro Viebrantz. Como desenvolver uma aplicação completa, desde o backend, passando por um app com push notifications e um pouquinho de internet das coisas com a Amazon Alexa. Tudo isso utilizando NodeJS e rodando de forma fácil e escalável no Google Cloud Platform
Backend, app e internet das coisas com NodeJS no Google Cloud PlatformAlvaro Viebrantz
This document discusses building backend systems, apps, and internet of things solutions using Node.js and Google Cloud Platform. It begins with an overview of cloud computing and GCP before demonstrating a sample project that includes a Node.js backend using Google App Engine, mobile apps using React Native and Firebase, and an Alexa skill. It also covers using Datastore for storage, scheduling tasks with cron jobs, push notifications, and serverless functions.
In the last sessions we have seen that P4D (Python 4 Delphi) is powerful enough to offer components, Python packages or libraries in Delphi or Lazarus (FPC). This time we go the other way of usage and integration; how does the Python or web world in the shell benefit from the VCL components as GUI controls. We create a Python extension module from Delphi classes, packages or functions. Building Delphi’s VCL library as a specific Python module in a console or editor and launching a complete Windows GUI from a script can be the start of a long journey.
The flood of Open APIs is now so blatant that we take a closer look at some basics and principles. Of course, the best way to understand how APIs work is to try them. While most APIs require access via API keys or have complicated authentication and authorization methods, there are also open APIs with no requirements or licenses whatsoever. This is especially useful for beginners as we can start exploring different APIs right away. It’s also useful for web developers who want easy access to a sample dataset for their app; e.g. most weather apps get their weather forecast data from a weather API instead of building weather stations themselves.
Faker is a Python library that generates fake data. Fake data is often used for testing or filling databases with some dummy data. Faker is heavily inspired by PHP's Faker, Perl's Data::Faker, and by Ruby's Faker.
Many of the applications and organizations provide avatar features. Finally, synthetic datasets can minimize privacy concerns. Attempts to anonymize data can be ineffective, as even if sensitive/identifying variables are removed from the dataset
Python for Delphi (P4D) is a set of free components that wrap up the Python DLL into Delphi and Lazarus (FPC). They let you easily execute Python scripts, create new Python modules and new Python types. You can create Python extensions as DLLs and much more like scripting. P4D provides different levels of functionality: Low-level access to the python API High-level bi-directional interaction with Python Access to Python objects using Delphi custom variants (VarPyth.pas).
Python for Delphi (P4D) is a set of free components that wrap up the Python DLL into Delphi and Lazarus (FPC). They let you easily execute Python scripts, create new Python modules and new Python types. You can create Python extensions as DLLs and much more like scripting. P4D provides different levels of functionality:
Low-level access to the python API
High-level bi-directional interaction with Python
Access to Python objects using Delphi custom variants (VarPyth.pas)
Wrapping of Delphi objects for use in python scripts using RTTI (WrapDelphi.pas)
Creating python extension modules with Delphi classes and functions
Generate Scripts in maXbox from Python Installation
The document describes steps to build and train an image classification model using Lazarus, the neural-api library, and Google Colab. It clones the neural-api GitHub repository, installs dependencies like FPC and Lazarus, builds and trains a simple image classifier on the CIFAR-10 dataset, and exports the trained model weights and training logs. The process demonstrates how to leverage Google Colab's GPUs to train deep learning models using Lazarus and Pascal.
The portable pixmap format(PPM), the portable graymap format(PGM) and portable bitmap format(PBM) are image file formats designed to be easily exchanged between platforms. They are also sometimes referred collectively as the portable anymap format(PNM). These formats are a convenient (simple) method of saving image data. And the format is not even limited to graphics, its definition allowing it to be used for arbitrary three-dimensional matrices or cubes of unsigned integers.
This tutor puts a trip to the kingdom of object recognition with computer vision knowledge and an image classifier.
Object detection has been witnessing a rapid revolutionary change in some fields of computer vision. Its involvement in the combination of object classification
as well as object recognition makes it one of the most challenging topics in the domain of machine learning & vision.
How can we visualize data in machine learning with VS Code? This is a C# wrapper for the GraphViz graph generator for dotnet core. Further bindings for Python GraphViz are shown and exports to MS Power BI all in MS Visual Code, Jupyter and dotnet core.
K-CAI NEURAL API is a Keras based neural network API for machine learning that will allow you to prototype with a lots of possibilities of Tensorflow! Python, Free Pascal and Delphi together in Google Colab, Git or the Community Edition.
Software is changing the world. CGC is a Common Gateway Coding as the name says, it is a "common" language approach for almost everything. I want to show how a multi-language approach to infrastructure as code using general purpose programming languages lets cloud engineers and code producers unlocking the same software engineering techniques commonly used for applications.
Code Review Checklist: How far is a code review going? "Metrics measure the design of code after it has been written, a Review proofs it and Refactoring improves code."
In this paper a document structure is shown and tips for a code review.
Some checks fits with your existing tools and simply raises a hand when the quality or security of your codebase is impaired.
Open LDAP as A directory serviceis a system for storing and retrieving information in a tree-like structure with the following key properties:
Optimized for reading Distributed storage model Extensible data storage types Advanced search capabilities Consistent replication possibilities
This document discusses closures and functional programming. It begins with an agenda that covers closures as code blocks, their history in languages like Lisp and Scheme, examples of functional programming, and using closures for refactoring. It then discusses a case study on experiences with a polygraph design, including optimizations with closures, packaging, and applying the Demeter principle. Finally, it provides links for further reading on closures.
This document explains how to redirect console output from a GUI application to the parent command prompt process. It describes using the AttachConsole and FreeConsole functions to attach and detach a process from the console. The GetParentProcessName function is used to get the name of the parent process (e.g. cmd.exe or powershell.exe) to determine if output should be redirected. The code sample shows attaching the console, writing sample output, and detaching when complete.
Introduction to use machine learning in python and pascal to do such a thing like train prime numbers when there are algorithms in place to determine prime numbers. See a dataframe, feature extracting and a few plots to re-search for another hot experiment to predict prime numbers.
This tutor shows the train and test set split with binary classifying, clustering and 3D plots and discuss a probability density function in scikit-learn on synthetic datasets. The dataset is very simple as a reference of understanding.
This document discusses machine learning techniques including linear support vector machines (SVMs), data splitting, model fitting and prediction, and histograms. It summarizes an SVM tutorial for predicting samples and evaluating models using classification reports and confusion matrices. It also covers kernel density estimation, PCA, and comparing different classifiers.
In this article you will learn hot to use tensorflow Softmax Classifier estimator to classify MNIST dataset in one script.
This paper introduces also the basic idea of a artificial neural network.
The term “machine learning” is used to describe one kind of “artificial intelligence” (AI) where a machine is able to learn and adapt through its own experience. We crawled and collected 30 top overview diagrams which shows the topic of methods, algorithms and concepts.
Prediction of Electrical Energy Efficiency Using Information on Consumer's Ac...PriyankaKilaniya
Energy efficiency has been important since the latter part of the last century. The main object of this survey is to determine the energy efficiency knowledge among consumers. Two separate districts in Bangladesh are selected to conduct the survey on households and showrooms about the energy and seller also. The survey uses the data to find some regression equations from which it is easy to predict energy efficiency knowledge. The data is analyzed and calculated based on five important criteria. The initial target was to find some factors that help predict a person's energy efficiency knowledge. From the survey, it is found that the energy efficiency awareness among the people of our country is very low. Relationships between household energy use behaviors are estimated using a unique dataset of about 40 households and 20 showrooms in Bangladesh's Chapainawabganj and Bagerhat districts. Knowledge of energy consumption and energy efficiency technology options is found to be associated with household use of energy conservation practices. Household characteristics also influence household energy use behavior. Younger household cohorts are more likely to adopt energy-efficient technologies and energy conservation practices and place primary importance on energy saving for environmental reasons. Education also influences attitudes toward energy conservation in Bangladesh. Low-education households indicate they primarily save electricity for the environment while high-education households indicate they are motivated by environmental concerns.
Build the Next Generation of Apps with the Einstein 1 Platform.
Rejoignez Philippe Ozil pour une session de workshops qui vous guidera à travers les détails de la plateforme Einstein 1, l'importance des données pour la création d'applications d'intelligence artificielle et les différents outils et technologies que Salesforce propose pour vous apporter tous les bénéfices de l'IA.
Open Channel Flow: fluid flow with a free surfaceIndrajeet sahu
Open Channel Flow: This topic focuses on fluid flow with a free surface, such as in rivers, canals, and drainage ditches. Key concepts include the classification of flow types (steady vs. unsteady, uniform vs. non-uniform), hydraulic radius, flow resistance, Manning's equation, critical flow conditions, and energy and momentum principles. It also covers flow measurement techniques, gradually varied flow analysis, and the design of open channels. Understanding these principles is vital for effective water resource management and engineering applications.
Accident detection system project report.pdfKamal Acharya
The Rapid growth of technology and infrastructure has made our lives easier. The
advent of technology has also increased the traffic hazards and the road accidents take place
frequently which causes huge loss of life and property because of the poor emergency facilities.
Many lives could have been saved if emergency service could get accident information and
reach in time. Our project will provide an optimum solution to this draw back. A piezo electric
sensor can be used as a crash or rollover detector of the vehicle during and after a crash. With
signals from a piezo electric sensor, a severe accident can be recognized. According to this
project when a vehicle meets with an accident immediately piezo electric sensor will detect the
signal or if a car rolls over. Then with the help of GSM module and GPS module, the location
will be sent to the emergency contact. Then after conforming the location necessary action will
be taken. If the person meets with a small accident or if there is no serious threat to anyone’s
life, then the alert message can be terminated by the driver by a switch provided in order to
avoid wasting the valuable time of the medical rescue team.
A high-Speed Communication System is based on the Design of a Bi-NoC Router, ...DharmaBanothu
The Network on Chip (NoC) has emerged as an effective
solution for intercommunication infrastructure within System on
Chip (SoC) designs, overcoming the limitations of traditional
methods that face significant bottlenecks. However, the complexity
of NoC design presents numerous challenges related to
performance metrics such as scalability, latency, power
consumption, and signal integrity. This project addresses the
issues within the router's memory unit and proposes an enhanced
memory structure. To achieve efficient data transfer, FIFO buffers
are implemented in distributed RAM and virtual channels for
FPGA-based NoC. The project introduces advanced FIFO-based
memory units within the NoC router, assessing their performance
in a Bi-directional NoC (Bi-NoC) configuration. The primary
objective is to reduce the router's workload while enhancing the
FIFO internal structure. To further improve data transfer speed,
a Bi-NoC with a self-configurable intercommunication channel is
suggested. Simulation and synthesis results demonstrate
guaranteed throughput, predictable latency, and equitable
network access, showing significant improvement over previous
designs
3rd International Conference on Artificial Intelligence Advances (AIAD 2024)GiselleginaGloria
3rd International Conference on Artificial Intelligence Advances (AIAD 2024) will act as a major forum for the presentation of innovative ideas, approaches, developments, and research projects in the area advanced Artificial Intelligence. It will also serve to facilitate the exchange of information between researchers and industry professionals to discuss the latest issues and advancement in the research area. Core areas of AI and advanced multi-disciplinary and its applications will be covered during the conferences.
2. 2
Agenda EKON
• Get API Key (access token) of mapbox
• What's important in REST Calls?
• The Mapbox Static API
• Http Get & TCP Sockets
• 4-5 Use Cases
3. 3
Get a Key first
● All you need to start using Mapbox is an access token:
it's a short code that lets you use your Mapbox
account's capabilities or state.
● You can use the same access token with all interactive
mapping libraries, Python, Pascal and JavaScript
SDKs, and even directly against the REST APIs.
https://www.mapbox.com/developers/
4. 4
Important for REST Calls
Check HTTPS with Http Get()
• String Stream or Memory Stream ?
• UTF8ToString();
• Check Null or empty Values
• GeoJSON as String
• Try except / finally pattern
demo: 711_geo_satellite_mapbox_ekon20.txt
5. 5
Menus & Mouse are gone
Iterator Disable Structure
with MaxForm1.MainMenu1 do
for it:= 0 to items.count-1 do begin
TMenuItem(items).items[it].enabled:= false;
writeln(TMenuItem(items).items[it].name);
End;
http://www.gtro.com/delphi/parser_e.php
Demo: 547_Win10_Develop_4_BASTA_EKON19.TXT
6. 6
Intro: Static API
• Static API (classic)
• Get stand-alone images from classic
styles to embed on a form with the
option to add vector data as overlays.
→ Demo: Static, Geocoding, Distance, Satellite
• 711_geo_satellite_mapbox_ekon20.txt
• 692_GEO_fservermain33_EKON20.pas
• 692_GEOimageserv_fClient33_EKON20.pas
7. 7
HTTP versus TCP
TIdHTTPServer does not trigger an OnCommand... event
until it has read a complete HTTP request header from
the client.
A TIdTCPServer is not reading any requests at all!
A web browser would communicate with a TIdTCPServer
using the HTTP protocol, which only contains lines of
text in its request headers.
Thread: Indy https post returns HTTP/1.0 500 Error - help
https://forums.embarcadero.com/thread.jspa?threadID=112596
9. 9
Combine with Weather
• Check Services as Backup (ex. Geolocation)
• Call Contracts in an account (Access tokens)
• Call Services First CSF (map comp or service)
MyPublicIP:= GetHTM('http://ipinfo.io/ip');
function GetGeoWeather(const location: string;
const UrlRestAPI: string): string;
Demo: 640_rest_weather_report2.txt
10. 10
Finally you can test with
• Duplicated code in app & browser
• UWP Coding standards & guidelines
• Unit tests for Code Coverage
• Wine Reference or RealVNC on Android
• UISpy to find Components & Controls
• Comments & Compiler Directives
demo: 393_QRCode3.TXT
http://docs.codehaus.org/display/SONAR/Developers%27+Seven+Deadly+Sins