High level overview of Electron performance tips and tricks, and a look at the new C++ architecture of Mailspring (https://getmailspring.com/) — presented Nov. 1st at the Bay Area Electron Meetup
Benchmarking: You're Doing It Wrong (StrangeLoop 2014)Aysylu Greenberg
Knowledge of how to set up good benchmarks is invaluable in understanding performance of the system. Writing correct and useful benchmarks is hard, and verification of the results is difficult and prone to errors. When done right, benchmarks guide teams to improve the performance of their systems. When done wrong, hours of effort may result in a worse performing application, upset customers or worse! In this talk, we will discuss what you need to know to write better benchmarks. We will look at examples of bad benchmarks and learn about what biases can invalidate the measurements, in the hope of correctly applying our new-found skills and avoiding such pitfalls in the future.
Make Data Smart Again 2018 - Building a Smart Security Camera with Raspberry ...Mark West
In this session I’ll share the story of how I transformed a simple Raspberry Pi Zero webcam into a Smart Security Camera (with motion detection, threat detection and alert notifications) by combining open source software with cloud based AI. Attendees can expect a demonstration of how I used a range of AWS API’s (including Rekognition, Lambda and Step Functions) to help my Smart Security Camera distinguish between an unwanted guest and the neighbours’ cat.
The document discusses cloud computing and sample cloud deployment architectures. It provides examples of how Animoto used RightScale to automatically scale their infrastructure in response to increased traffic. It also describes strategies for database failover and high availability in the cloud, including promoting a slave database to master if the primary fails and starting replication from snapshots.
The document discusses cloud computing and sample cloud deployment architectures. It describes how Animoto used RightScale to automatically scale their infrastructure in response to increased load. It also addresses how to handle database failures in the cloud by promoting a slave database to master and launching a new slave from a snapshot when the master fails.
How does the guardian website scale?
With millions of page views per month, we need to think about scaling to an extreme level. But being Agile we did it as we went.
Updating Ember Models in Real-time with Sockets and RxKeith Silgard
1) The document discusses updating an Ember application in real-time by connecting to a live events socket to receive a mixed feed of all subscribed events and manually register for chat channels.
2) An initial attempt at going real-time had issues with staying connected to an unpredictable service, updating with messages sent while disconnected, and maintaining state on live chat channels.
3) RxJS was used as a solution but introduced complexity, though it transformed observable sequences into the most recent one and addressed issues like staying connected and updating models.
AWS Lambda is a serverless compute service that runs code in response to events, automatically managing resources. It handles capacity, scaling, deployment, monitoring, logging, and security patching. Developers pay only for the compute time used to run their code, avoiding the need to provision and manage servers. Lambda functions can be triggered by events from services like API Gateway or S3 and are written in languages like Node.js, Python, Java, and C#.
Benchmarking: You're Doing It Wrong (StrangeLoop 2014)Aysylu Greenberg
Knowledge of how to set up good benchmarks is invaluable in understanding performance of the system. Writing correct and useful benchmarks is hard, and verification of the results is difficult and prone to errors. When done right, benchmarks guide teams to improve the performance of their systems. When done wrong, hours of effort may result in a worse performing application, upset customers or worse! In this talk, we will discuss what you need to know to write better benchmarks. We will look at examples of bad benchmarks and learn about what biases can invalidate the measurements, in the hope of correctly applying our new-found skills and avoiding such pitfalls in the future.
Make Data Smart Again 2018 - Building a Smart Security Camera with Raspberry ...Mark West
In this session I’ll share the story of how I transformed a simple Raspberry Pi Zero webcam into a Smart Security Camera (with motion detection, threat detection and alert notifications) by combining open source software with cloud based AI. Attendees can expect a demonstration of how I used a range of AWS API’s (including Rekognition, Lambda and Step Functions) to help my Smart Security Camera distinguish between an unwanted guest and the neighbours’ cat.
The document discusses cloud computing and sample cloud deployment architectures. It provides examples of how Animoto used RightScale to automatically scale their infrastructure in response to increased traffic. It also describes strategies for database failover and high availability in the cloud, including promoting a slave database to master if the primary fails and starting replication from snapshots.
The document discusses cloud computing and sample cloud deployment architectures. It describes how Animoto used RightScale to automatically scale their infrastructure in response to increased load. It also addresses how to handle database failures in the cloud by promoting a slave database to master and launching a new slave from a snapshot when the master fails.
How does the guardian website scale?
With millions of page views per month, we need to think about scaling to an extreme level. But being Agile we did it as we went.
Updating Ember Models in Real-time with Sockets and RxKeith Silgard
1) The document discusses updating an Ember application in real-time by connecting to a live events socket to receive a mixed feed of all subscribed events and manually register for chat channels.
2) An initial attempt at going real-time had issues with staying connected to an unpredictable service, updating with messages sent while disconnected, and maintaining state on live chat channels.
3) RxJS was used as a solution but introduced complexity, though it transformed observable sequences into the most recent one and addressed issues like staying connected and updating models.
AWS Lambda is a serverless compute service that runs code in response to events, automatically managing resources. It handles capacity, scaling, deployment, monitoring, logging, and security patching. Developers pay only for the compute time used to run their code, avoiding the need to provision and manage servers. Lambda functions can be triggered by events from services like API Gateway or S3 and are written in languages like Node.js, Python, Java, and C#.
Scaling on Amazon AWS : From the perspective of AWS, and the application stack. Talks about the available options on AWS, and also the architecture of the scalable application.
What is test-driven Infrastructure ? How you can test your ops side ? What is chaos monkey and how Sony is using it ? What is testinfra ? How to write acceptance tests in Infrastructure ?
The challenges of live events scalabilityGuy Tomer
Guy Tomer, founder of attracTV, discusses the challenges of supporting large online live events with TV participation. AttracTV supported an online music awards show with over 1 million streams and 100,000 concurrent users. Key challenges included scaling to support these large user numbers, handling steep increases in users, managing large amounts of data, maintaining high availability, testing at this scale, and controlling costs. Tomer emphasizes planning for horizontal scalability, keeping things simple, testing under realistic conditions, continuous monitoring, and using cloud infrastructure to handle demand flexibly at an affordable cost.
Spotify's Music Recommendations Lambda ArchitectureEsh Vckay
Spotify's Collaborative Filtering platform powers our Discover Page. In light of the fact, that we have been adding one new user every three seconds, it is paramount that we do things in real-time. We redesigned our recommendation system and added a Storm based real-time platform.
This document discusses myTaxi's use of AWS cloud computing services. It describes myTaxi as a taxi booking service that uses Socket Communication, Java, PostgreSQL, Apache, PHP, EC2 instances, CloudFront for images, and plans to use Elastic Beanstalk. Some benefits of AWS are heavy load scaling, pay-as-you-use model, fast scaling, and international availability. Some problems are lack of custom hardware, slow EBS for databases, single IP per instance, slow elastic IP takeover, and instance retirement scheduling issues.
Denver AWS Users' Group Meeting - July 2018 SlidesDavid McDaniel
This document summarizes the July 2018 meeting of the AWS Users' Group in Denver. It thanks sponsors Eplexity and Prestige for hosting the event and providing food and drinks. The agenda includes discussions on methodology changes when moving applications to the cloud as well as cloud optimization best practices. Attendees are provided WiFi login details and reminded to follow the group on Twitter. Upcoming events and resources are also listed.
This document outlines 5 challenges that can be addressed using new features in FME Server 2019. It summarizes each challenge and demonstrates how automations and system events in FME Server can be used to automatically run workspaces when data arrives, run nightly updates on a schedule, orchestrate complex workflows, and notify administrators via email when new users are created. It concludes by listing some of the new features in FME Server 2019 such as FME Server apps, automations, system events, job log filtering, and improved password security rules and user preferences.
Monitoring the unknown, 1000*100 series a day - Big Data Vilnius 2017Quentin Adam
How to monitor unknown third party code? One of the hardest challenges we face running Clever Cloud, apart from the impressive scale we face with hundreds of new applications per week, is the monitoring of unknown tech stacks. The first goal of rebuilding the monitoring platform was to accommodate the immutable infrastructure pattern that generates lots of ephemeral hosts every minute. The traditional approach is to focus on VMs or hosts, not applications. We needed to shift this into an approach of auto-discovery of metrics to monitor, allowing third party code to publish new items. This talk explains our journey in building Clever Cloud Metrics stack, heavily based on Warp10 (Kafka/Hadoop/Storm based) to deliver developer efficiency and trustability to our clients applications.
JUST EAT: Tools we use to enable our culturePeter Mounce
Just Eat is an online takeaway company that has been operating since 2001 in Denmark and 2005 in the UK. They have adopted a DevOps culture where each team owns their own features and infrastructure. To enable this distributed system on Windows at scale without an Active Directory, they rely on tools like persistent chat, real-time monitoring, alerting, and centralized logging to improve collaboration and debugging in production. Their most important aspects are culture, autonomy for teams, and pragmatic use of tools based on merit.
Tavernalc: How to transform your OpenOffice Calc into a grid.Alexandro Colorado
This document discusses Tavernalc, a tool that integrates the workflow capabilities of Taverna into OpenOffice Calc spreadsheets. It provides an overview of Tavernalc's functionality from the user and developer perspectives, how it represents Taverna workflows and data types within Calc, and the developers' experience using the OpenOffice API.
Promise of a better future by Rahul Goma Phulore and Pooja Akshantal, Thought...Thoughtworks
With the recent, vivid trend towards multicore hardware and the ever growing application requirements, concurrency is no more a niche area it used to be, and is slowly becoming a norm. In this talk, we will talk about promises/futures, one of the concurrency models that has risen to the occasion. We will look at what they are, how they're implemented and used in Java and Javascript. We will see how Scala, with its functional paradigm and greater abstraction capabilities, avoids "callback hell" typically associated with the model, allows writing of concurrent code in "direct style", and thereby greatly reduces the cognitive burden, allowing you to focus on application logic better.
This document discusses serverless computing with AWS Lambda. It begins by explaining that serverless does not mean there are no servers, but rather code is executed when triggered by events. An example is given of a serverless API using API Gateway to trigger a Lambda function. The document outlines some key aspects of how serverless works with Lambda such as supported languages, logging to CloudWatch, and resource limits for Lambda functions. Overall limits are discussed and it is noted autoscaling can provide more resources when needed. The document concludes by mentioning potential use cases for serverless and posing questions.
Gilles will be sharing some of the experiences they faced first hand, how they resolved them and what they are looking to do differently in the future
* Presented at the Sydney AWS Meetup session 6th July 2016
http://www.meetup.com/AWS-Sydney/
Hosted and organised by PolarSeven - http://polarseven.com
Technical presentation around the challenges and technical concerns, you need to plan when you try to make the jump of using containers in a production environment with Amazon Elastic Container Service (ECS)
The document discusses using FME 2019 to help the FME Lizard find a new office location based on specific criteria. It then summarizes the key highlights of FME 2019 Desktop including improved performance, a dark mode, new transformers, readers/writers for additional formats, and the ability to more quickly obtain fixes and new features through FME packages. The document concludes by noting upcoming FME connectors and mobile app capabilities in FME 2019.
This document is Charles Qian's resume. It summarizes that he is a computer science student at Cornell University expected to graduate in May 2017 with a 3.72 GPA. It lists his relevant coursework, skills in programming languages and tools, experience as an intern and teaching assistant, and projects including open source contributions and hackathon projects. It also notes that he is currently president of his a cappella group.
Quarterly Technology Briefing, Manchester, UK September 2013Thoughtworks
This document summarizes a tech lab experiment conducted by ThoughtWorks to explore graph databases and cloud deployment. The team modeled Manchester's tram network as a graph database to understand its benefits. They iteratively improved the data and query models. The working app was deployed to AWS and automatically updated. The team successfully met the desired outcomes of the lab by gaining technical knowledge, producing a working demo, collateral materials, and excitement in the community. Next steps discussed expanding the work to clients and open sourcing parts of the project.
The author explains continuous integration to their parents by describing how they automated the provisioning of cloud infrastructure on Amazon Web Services (AWS) using configuration files, source control, and a Jenkins continuous integration system. The Jenkins project checks out configuration files from source control, manipulates them to deploy AWS resources like servers and databases, and allows developers to build test environments by clicking a green button, saving significant time compared to manual deployment.
How to build a SaaS solution in 60 daysBrett McLain
Brett McLain built a SaaS solution in 60 days using MeteorJS to manage remote devices that customers used to control things like vehicle starters and irrigation pumps remotely via SMS or phone calls. He chose MeteorJS because it provided an integrated full-stack framework with reactivity out of the box. Key features included authentication using accounts packages, managing user devices and commands in MongoDB, and integrating Twilio and Stripe for SMS/call functionality and payments. The final application allowed users to control devices, monitor usage, and subscribe to the service with a credit card through Stripe for a simple backend-free payment experience.
Lessons Learned Replatforming A Large Machine Learning Application To Apache ...Databricks
Morningstar’s Risk Model project is created by stitching together statistical and machine learning models to produce risk and performance metrics for millions of financial securities. Previously, we were running a single version of this application, but needed to expand it to allow for customizations based on client demand. With the goal of running hundreds of custom Risk Model runs at once at an output size of around 1TB of data each, we had a challenging technical problem on our hands! In this presentation, we’ll talk about the challenges we faced replatforming this application to Spark, how we solved them, and the benefits we saw.
Some things we’ll touch on include how we created customized models, the architecture of our machine learning application, how we maintain an audit trail of data transformations (for rigorous third party audits), and how we validate the input data our model takes in and output data our model produces. We want the attendees to walk away with some key ideas of what worked for us when productizing a large scale machine learning platform.
Scaling on Amazon AWS : From the perspective of AWS, and the application stack. Talks about the available options on AWS, and also the architecture of the scalable application.
What is test-driven Infrastructure ? How you can test your ops side ? What is chaos monkey and how Sony is using it ? What is testinfra ? How to write acceptance tests in Infrastructure ?
The challenges of live events scalabilityGuy Tomer
Guy Tomer, founder of attracTV, discusses the challenges of supporting large online live events with TV participation. AttracTV supported an online music awards show with over 1 million streams and 100,000 concurrent users. Key challenges included scaling to support these large user numbers, handling steep increases in users, managing large amounts of data, maintaining high availability, testing at this scale, and controlling costs. Tomer emphasizes planning for horizontal scalability, keeping things simple, testing under realistic conditions, continuous monitoring, and using cloud infrastructure to handle demand flexibly at an affordable cost.
Spotify's Music Recommendations Lambda ArchitectureEsh Vckay
Spotify's Collaborative Filtering platform powers our Discover Page. In light of the fact, that we have been adding one new user every three seconds, it is paramount that we do things in real-time. We redesigned our recommendation system and added a Storm based real-time platform.
This document discusses myTaxi's use of AWS cloud computing services. It describes myTaxi as a taxi booking service that uses Socket Communication, Java, PostgreSQL, Apache, PHP, EC2 instances, CloudFront for images, and plans to use Elastic Beanstalk. Some benefits of AWS are heavy load scaling, pay-as-you-use model, fast scaling, and international availability. Some problems are lack of custom hardware, slow EBS for databases, single IP per instance, slow elastic IP takeover, and instance retirement scheduling issues.
Denver AWS Users' Group Meeting - July 2018 SlidesDavid McDaniel
This document summarizes the July 2018 meeting of the AWS Users' Group in Denver. It thanks sponsors Eplexity and Prestige for hosting the event and providing food and drinks. The agenda includes discussions on methodology changes when moving applications to the cloud as well as cloud optimization best practices. Attendees are provided WiFi login details and reminded to follow the group on Twitter. Upcoming events and resources are also listed.
This document outlines 5 challenges that can be addressed using new features in FME Server 2019. It summarizes each challenge and demonstrates how automations and system events in FME Server can be used to automatically run workspaces when data arrives, run nightly updates on a schedule, orchestrate complex workflows, and notify administrators via email when new users are created. It concludes by listing some of the new features in FME Server 2019 such as FME Server apps, automations, system events, job log filtering, and improved password security rules and user preferences.
Monitoring the unknown, 1000*100 series a day - Big Data Vilnius 2017Quentin Adam
How to monitor unknown third party code? One of the hardest challenges we face running Clever Cloud, apart from the impressive scale we face with hundreds of new applications per week, is the monitoring of unknown tech stacks. The first goal of rebuilding the monitoring platform was to accommodate the immutable infrastructure pattern that generates lots of ephemeral hosts every minute. The traditional approach is to focus on VMs or hosts, not applications. We needed to shift this into an approach of auto-discovery of metrics to monitor, allowing third party code to publish new items. This talk explains our journey in building Clever Cloud Metrics stack, heavily based on Warp10 (Kafka/Hadoop/Storm based) to deliver developer efficiency and trustability to our clients applications.
JUST EAT: Tools we use to enable our culturePeter Mounce
Just Eat is an online takeaway company that has been operating since 2001 in Denmark and 2005 in the UK. They have adopted a DevOps culture where each team owns their own features and infrastructure. To enable this distributed system on Windows at scale without an Active Directory, they rely on tools like persistent chat, real-time monitoring, alerting, and centralized logging to improve collaboration and debugging in production. Their most important aspects are culture, autonomy for teams, and pragmatic use of tools based on merit.
Tavernalc: How to transform your OpenOffice Calc into a grid.Alexandro Colorado
This document discusses Tavernalc, a tool that integrates the workflow capabilities of Taverna into OpenOffice Calc spreadsheets. It provides an overview of Tavernalc's functionality from the user and developer perspectives, how it represents Taverna workflows and data types within Calc, and the developers' experience using the OpenOffice API.
Promise of a better future by Rahul Goma Phulore and Pooja Akshantal, Thought...Thoughtworks
With the recent, vivid trend towards multicore hardware and the ever growing application requirements, concurrency is no more a niche area it used to be, and is slowly becoming a norm. In this talk, we will talk about promises/futures, one of the concurrency models that has risen to the occasion. We will look at what they are, how they're implemented and used in Java and Javascript. We will see how Scala, with its functional paradigm and greater abstraction capabilities, avoids "callback hell" typically associated with the model, allows writing of concurrent code in "direct style", and thereby greatly reduces the cognitive burden, allowing you to focus on application logic better.
This document discusses serverless computing with AWS Lambda. It begins by explaining that serverless does not mean there are no servers, but rather code is executed when triggered by events. An example is given of a serverless API using API Gateway to trigger a Lambda function. The document outlines some key aspects of how serverless works with Lambda such as supported languages, logging to CloudWatch, and resource limits for Lambda functions. Overall limits are discussed and it is noted autoscaling can provide more resources when needed. The document concludes by mentioning potential use cases for serverless and posing questions.
Gilles will be sharing some of the experiences they faced first hand, how they resolved them and what they are looking to do differently in the future
* Presented at the Sydney AWS Meetup session 6th July 2016
http://www.meetup.com/AWS-Sydney/
Hosted and organised by PolarSeven - http://polarseven.com
Technical presentation around the challenges and technical concerns, you need to plan when you try to make the jump of using containers in a production environment with Amazon Elastic Container Service (ECS)
The document discusses using FME 2019 to help the FME Lizard find a new office location based on specific criteria. It then summarizes the key highlights of FME 2019 Desktop including improved performance, a dark mode, new transformers, readers/writers for additional formats, and the ability to more quickly obtain fixes and new features through FME packages. The document concludes by noting upcoming FME connectors and mobile app capabilities in FME 2019.
This document is Charles Qian's resume. It summarizes that he is a computer science student at Cornell University expected to graduate in May 2017 with a 3.72 GPA. It lists his relevant coursework, skills in programming languages and tools, experience as an intern and teaching assistant, and projects including open source contributions and hackathon projects. It also notes that he is currently president of his a cappella group.
Quarterly Technology Briefing, Manchester, UK September 2013Thoughtworks
This document summarizes a tech lab experiment conducted by ThoughtWorks to explore graph databases and cloud deployment. The team modeled Manchester's tram network as a graph database to understand its benefits. They iteratively improved the data and query models. The working app was deployed to AWS and automatically updated. The team successfully met the desired outcomes of the lab by gaining technical knowledge, producing a working demo, collateral materials, and excitement in the community. Next steps discussed expanding the work to clients and open sourcing parts of the project.
The author explains continuous integration to their parents by describing how they automated the provisioning of cloud infrastructure on Amazon Web Services (AWS) using configuration files, source control, and a Jenkins continuous integration system. The Jenkins project checks out configuration files from source control, manipulates them to deploy AWS resources like servers and databases, and allows developers to build test environments by clicking a green button, saving significant time compared to manual deployment.
How to build a SaaS solution in 60 daysBrett McLain
Brett McLain built a SaaS solution in 60 days using MeteorJS to manage remote devices that customers used to control things like vehicle starters and irrigation pumps remotely via SMS or phone calls. He chose MeteorJS because it provided an integrated full-stack framework with reactivity out of the box. Key features included authentication using accounts packages, managing user devices and commands in MongoDB, and integrating Twilio and Stripe for SMS/call functionality and payments. The final application allowed users to control devices, monitor usage, and subscribe to the service with a credit card through Stripe for a simple backend-free payment experience.
Lessons Learned Replatforming A Large Machine Learning Application To Apache ...Databricks
Morningstar’s Risk Model project is created by stitching together statistical and machine learning models to produce risk and performance metrics for millions of financial securities. Previously, we were running a single version of this application, but needed to expand it to allow for customizations based on client demand. With the goal of running hundreds of custom Risk Model runs at once at an output size of around 1TB of data each, we had a challenging technical problem on our hands! In this presentation, we’ll talk about the challenges we faced replatforming this application to Spark, how we solved them, and the benefits we saw.
Some things we’ll touch on include how we created customized models, the architecture of our machine learning application, how we maintain an audit trail of data transformations (for rigorous third party audits), and how we validate the input data our model takes in and output data our model produces. We want the attendees to walk away with some key ideas of what worked for us when productizing a large scale machine learning platform.
Jeremy Edberg (MinOps ) - How to build a solid infrastructure for a startup t...Startupfest
You're building your startup and you know it will be big. You don't want to spend a lot of time on infrastructure, but you also don't want to be putting out fires after you get mentioned on Hacker News. In this session, we will give you real practical tips that you can take home with you on building an infrastructure that will scale quickly with minimal up front work on your part, using time tested techniques in infrastructure as code, SaaS, and Serverless, among other things.
IBM and Node.js - Old Doge, New TricksDejan Glozic
The document discusses IBM's transition from older enterprise stacks like Java and SQL to newer technologies like Node.js, NoSQL, and microservices. It describes motivations for the change like yearly release cycles, large teams, and running products on-premise. It then covers specifics of the transition, including moving from Dojo/Dijit to jQuery/Bootstrap on the client side, using Node.js and WebSockets for real-time updates, adopting a microservices architecture, switching from SQL to NoSQL databases, and addressing challenges that arise from the new approaches.
The Sanger Institute generates large amounts of genomic data and requires significant compute resources to analyze it. It has experimented with running its analysis pipelines in the cloud to expand capacity and markets. However, moving large datasets into the cloud and ensuring fast access to the data within cloud compute resources has proved challenging. While individual components like web services have worked well, the high performance computing workloads that rely on large-scale data access and processing have not scaled effectively due to data transfer bottlenecks and lack of high-performance filesystems in the cloud.
Building a system for machine and event-oriented data with RocanaTreasure Data, Inc.
In this session, we’ll follow the flow of data through an end-to-end system built to handle tens of terabytes an hour of event-oriented data, providing real-time streaming, in-memory, SQL, and batch access to this data. We’ll go into detail on how open source systems such as Hadoop, Kafka, Solr, and Impala/Hive can be stitched together to form the base platform; describe how and where to perform data transformation and aggregation; provide a simple and pragmatic way of managing event metadata; and talk about how applications built on top of this platform get access to data and extend its functionality. Finally, a brief demo of Rocana Ops, an application for large scale data center operations, will be given, along with an explanation about how it uses the underlying platform.
The document discusses optimizing performance for Ajax applications. It recommends:
- Keeping client-side code light by only requesting necessary data from the server via JSON messages.
- Avoiding unnecessary DOM touches and reflows which are computationally expensive.
- Measuring performance before and after optimizations to validate improvements rather than relying on intuition.
- Optimizing algorithms and avoiding unnecessary work rather than prematurely optimizing without evidence of need.
Scaling for Success: Lessons from handling peak loads on Azure with NServiceBusParticular Software
What happens when 200k users unexpectedly decide to use your platform simultaneously? We’re using autoscale on Azure PaaS so surely we can handle that, right? Wrong! Ask me how I found out… After going through a bit of trouble, I want to help you avoid the same mistakes I made.
Building a system for machine and event-oriented data - Data Day Seattle 2015Eric Sammer
The document discusses building a system for machine data and event-oriented data. It describes the speaker's background and company, Rocana, which builds systems to operate modern data centers. The system ingests over 100k events per second, provides low latency and full data retention, and is used for tasks like quality of service monitoring, fraud detection, and security. It models all data as timestamped events and uses Kafka, consumers, and SQL for aggregation to power analytics and searches. The summary discusses key aspects of the system's architecture, guarantees, data modeling, and analytics capabilities.
Slides from my last presentation at the Cape Town Meteor meetup, on optimising the UI, specifically for Hybrid apps and for Meteor JS hybrid apps.
The main thrust is really more about design patterns, and carefully controlling data management in your mobile app, with great examples of these patterns out in the real world.
see the mobile patterns video here : https://www.youtube.com/watch?v=e6WWX4TF3UI
Serverless architectures let you build and deploy applications and services with infrastructure resources that require zero administration. In the past, you had to provision and scale servers to run your application code, install and operate distributed databases, and build and run custom software to handle API requests. Now, AWS provides a stack of scalable, fully-managed services that eliminates these operational complexities.
In this session, you will learn about the benefits of serverless architectures and the basics of the serverless stack AWS provides. We will also walk through how you can use serverless architectures for everything from data processing to mobile and web backends.
AWS DevDay San Francisco, June 21, 2016.
Presenter: Jeremy Edberg, Co-Founder, CloudNative, & AWS Community Hero
How and why we evolved a legacy Java web application to Scala... and we are s...Katia Aresti
Applications get old, and technology moves fast. Overtime, adding or modifying functionalities might become as expensive as re-coding everything all from scratch. But rewriting a complete website and its functionalities it’s hard if we want to minimize the risks of breaking existing functionalities and specially when this application fits in a ecosystem and interacts with other pieces of software and teams.
In this session, you will learn how we moved from a legacy java monolithic website using scala PlayFramework, AngularJS, Elasticsearch and MongoDB, how we built a multi service and REST oriented architecture, which were the technical and human problems we encountered and how we managed to solved them.
Building a system for machine and event-oriented data - Velocity, Santa Clara...Eric Sammer
This talk was presented at O'Reilly's Velocity conference in Santa Clara, May 28 2015.
Abstract: http://velocityconf.com/devops-web-performance-2015/public/schedule/detail/42284
This document discusses the need for continuous delivery in software development. It defines continuous delivery as making sure software can be reliably released at any time. The document outlines some key aspects of continuous delivery including automated testing, infrastructure as code, continuous integration, and blue/green deployments. It provides an example of implementing continuous delivery for a large retail company using tools like Jenkins, Puppet, Logstash and practices like infrastructure as code and automated testing.
2017 Melbourne YOW! CTO Summit - Monolith to micro-services with CQRS & Event...Douglas English
These are the slides I presented at the Melbourne 2017 YOW! CTO Summit. The talk followed Culture Amp's steps and miss-steps on our journey to micro-services, and how we came to find CQRS and Event Sourcing fantastic tools.
Parallel and Asynchronous Programming - ITProDevConnections 2012 (Greek)Panagiotis Kanavos
This document discusses parallel and asynchronous programming using the Task Parallel Library (TPL) in .NET. It covers how processors are getting smaller so parallelism is important. It provides examples of using TPL for data parallelism by partitioning work over collections and task parallelism by breaking work into steps. It also discusses asynchronous programming with async/await and how TPL handles cancellation, progress reporting, and synchronization contexts.
The document discusses how to build real-time web applications. It emphasizes storing data client-side and pushing changes to clients in real-time using technologies like websockets to avoid slow AJAX calls. Examples are given of building an auto-complete search and news ticker in a real-time manner by initially loading all data and then querying client-side or pushing updates via websockets. The key is avoiding roundtrips to databases by doing work client-side and using fast databases like Redis that support publishing changes.
Tech for the Non Technical - Anatomy of an Application StackIntelligent_ly
Building technology is a practiced skill and indeed an art, but it's not magic. You hire craftsmen and you trust them with the details. But in order to command their respect and sleep well at night, you deserve to have a big-picture understanding of what they're building and why. A little knowledge will go a long way towards confidently leading your technical product team.
Healthcare systems around the world are looking to Precision Medicine -- care decisions tailored for the individual patient -- as a means to drive better care outcomes at lower cost. Today, the most promising technology that has made this possible in certain diseases like cancer is sequencing a patient's genome. For infectious diseases, sequencing has revolutionized our understanding of outbreaks and how they spread. Genome sequencing has progressed significantly in the past decade to improve throughput and lower costs by 100X or more. It is a data and compute intensive endeavor, which most biomedical research and care delivery networks are not equipped to handle. This session features Dr. Swaine Chen from the Genome Institute of Singapore, and the Broad Institute Cromwell team, discussing the problem of dealing with the scale of genomic data, and how they solved these to deliver results.
Similar to Electron performance and C++ in Mailspring (20)
14 th Edition of International conference on computer visionShulagnaSarkar2
About the event
14th Edition of International conference on computer vision
Computer conferences organized by ScienceFather group. ScienceFather takes the privilege to invite speakers participants students delegates and exhibitors from across the globe to its International Conference on computer conferences to be held in the Various Beautiful cites of the world. computer conferences are a discussion of common Inventions-related issues and additionally trade information share proof thoughts and insight into advanced developments in the science inventions service system. New technology may create many materials and devices with a vast range of applications such as in Science medicine electronics biomaterials energy production and consumer products.
Nomination are Open!! Don't Miss it
Visit: computer.scifat.com
Award Nomination: https://x-i.me/ishnom
Conference Submission: https://x-i.me/anicon
For Enquiry: Computer@scifat.com
WWDC 2024 Keynote Review: For CocoaCoders AustinPatrick Weigel
Overview of WWDC 2024 Keynote Address.
Covers: Apple Intelligence, iOS18, macOS Sequoia, iPadOS, watchOS, visionOS, and Apple TV+.
Understandable dialogue on Apple TV+
On-device app controlling AI.
Access to ChatGPT with a guest appearance by Chief Data Thief Sam Altman!
App Locking! iPhone Mirroring! And a Calculator!!
Unveiling the Advantages of Agile Software Development.pdfbrainerhub1
Learn about Agile Software Development's advantages. Simplify your workflow to spur quicker innovation. Jump right in! We have also discussed the advantages.
The Rising Future of CPaaS in the Middle East 2024Yara Milbes
Explore "The Rising Future of CPaaS in the Middle East in 2024" with this comprehensive PPT presentation. Discover how Communication Platforms as a Service (CPaaS) is transforming communication across various sectors in the Middle East.
UI5con 2024 - Bring Your Own Design SystemPeter Muessig
How do you combine the OpenUI5/SAPUI5 programming model with a design system that makes its controls available as Web Components? Since OpenUI5/SAPUI5 1.120, the framework supports the integration of any Web Components. This makes it possible, for example, to natively embed own Web Components of your design system which are created with Stencil. The integration embeds the Web Components in a way that they can be used naturally in XMLViews, like with standard UI5 controls, and can be bound with data binding. Learn how you can also make use of the Web Components base class in OpenUI5/SAPUI5 to also integrate your Web Components and get inspired by the solution to generate a custom UI5 library providing the Web Components control wrappers for the native ones.
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...kalichargn70th171
In today's fiercely competitive mobile app market, the role of the QA team is pivotal for continuous improvement and sustained success. Effective testing strategies are essential to navigate the challenges confidently and precisely. Ensuring the perfection of mobile apps before they reach end-users requires thoughtful decisions in the testing plan.
Unlock the Secrets to Effortless Video Creation with Invideo: Your Ultimate G...The Third Creative Media
"Navigating Invideo: A Comprehensive Guide" is an essential resource for anyone looking to master Invideo, an AI-powered video creation tool. This guide provides step-by-step instructions, helpful tips, and comparisons with other AI video creators. Whether you're a beginner or an experienced video editor, you'll find valuable insights to enhance your video projects and bring your creative ideas to life.
Microservice Teams - How the cloud changes the way we workSven Peters
A lot of technical challenges and complexity come with building a cloud-native and distributed architecture. The way we develop backend software has fundamentally changed in the last ten years. Managing a microservices architecture demands a lot of us to ensure observability and operational resiliency. But did you also change the way you run your development teams?
Sven will talk about Atlassian’s journey from a monolith to a multi-tenanted architecture and how it affected the way the engineering teams work. You will learn how we shifted to service ownership, moved to more autonomous teams (and its challenges), and established platform and enablement teams.
Consistent toolbox talks are critical for maintaining workplace safety, as they provide regular opportunities to address specific hazards and reinforce safe practices.
These brief, focused sessions ensure that safety is a continual conversation rather than a one-time event, which helps keep safety protocols fresh in employees' minds. Studies have shown that shorter, more frequent training sessions are more effective for retention and behavior change compared to longer, infrequent sessions.
Engaging workers regularly, toolbox talks promote a culture of safety, empower employees to voice concerns, and ultimately reduce the likelihood of accidents and injuries on site.
The traditional method of conducting safety talks with paper documents and lengthy meetings is not only time-consuming but also less effective. Manual tracking of attendance and compliance is prone to errors and inconsistencies, leading to gaps in safety communication and potential non-compliance with OSHA regulations. Switching to a digital solution like Safelyio offers significant advantages.
Safelyio automates the delivery and documentation of safety talks, ensuring consistency and accessibility. The microlearning approach breaks down complex safety protocols into manageable, bite-sized pieces, making it easier for employees to absorb and retain information.
This method minimizes disruptions to work schedules, eliminates the hassle of paperwork, and ensures that all safety communications are tracked and recorded accurately. Ultimately, using a digital platform like Safelyio enhances engagement, compliance, and overall safety performance on site. https://safelyio.com/
Measures in SQL (SIGMOD 2024, Santiago, Chile)Julian Hyde
SQL has attained widespread adoption, but Business Intelligence tools still use their own higher level languages based upon a multidimensional paradigm. Composable calculations are what is missing from SQL, and we propose a new kind of column, called a measure, that attaches a calculation to a table. Like regular tables, tables with measures are composable and closed when used in queries.
SQL-with-measures has the power, conciseness and reusability of multidimensional languages but retains SQL semantics. Measure invocations can be expanded in place to simple, clear SQL.
To define the evaluation semantics for measures, we introduce context-sensitive expressions (a way to evaluate multidimensional expressions that is consistent with existing SQL semantics), a concept called evaluation context, and several operations for setting and modifying the evaluation context.
A talk at SIGMOD, June 9–15, 2024, Santiago, Chile
Authors: Julian Hyde (Google) and John Fremlin (Google)
https://doi.org/10.1145/3626246.3653374
2. Me
• Mac & iOS ➡ React & Electron
• Run a small team doing React /
mobile dev, now working on
Mailspring.
• Led the team building Nylas
Mail from 2014–2016.
3. Nylas Mail ➡ Mailspring
• May 2017: Nylas API business booming—engineers
pulled from Nylas Mail to fight growth fires!
• Sept 2017: Nylas Mail officially sunset, re-licensed
under MIT. 🎉
Mailspring: Fix longstanding issues, use Pro version
revenue to run service and pay core contributors.
4.
5. ☄
Complaints about Nylas Mail
• Long startup time
• High battery impact
• Uses too much RAM*
Electron
😡 😤🚀🔥
6. • Long startup time
• High battery impact
• Uses too much RAM*
👷(
Complaints about Nylas Mail
Electron
7. Reducing startup time
• Start loading a BrowserWindow as fast as possible
• Watch out for sync I/O in main process
• Use the ASAR format in production
• Load less JavaScript (on launch)
14. Devtron
• Easy to find heavy dependencies and see total JS
loaded.
• `request` is 910k of JS. Can you use Fetch APIs?
`bluebird` is 180k of JS. Can you use native Promise?
Nylas Mail
15. Devtron
Mailspring
• Easy to find heavy dependencies and see total JS
loaded.
• `request` is 910k of JS. Can you use Fetch APIs?
`bluebird` is 180k of JS. Can you use native Promise?
16. Devtron
• Some things you can’t avoid… but you can load
later. Who needs spellcheck in the first 500ms?
More UI Plugins SpellcheckerCore Window
Interactive in 1.0s
20. Reducing Energy Impact
• Check for animations / needless layout
• Minimize use of setTimeout, setInterval. Don’t
wake and perform work if you don’t need to.
• Attach an event listener to `blur` that toggles
a CSS class, use it to prevent animations.
23. setTimeout / setInterval
• Use smarter polling strategies:
• Change polling rate based on Electron
PowerMonitor state.
• Poll when important events occur (window
focus event) rather than all the time.
24. But… what if it’s not enough?
• Need to sync mail every few minutes,
sometimes this requires polling.
• Need to keep IDLE connections open
to mail providers.
• Need to keep streaming connection
open to Mailspring server.
👷(😭 😢
27. Enter C++
• Command line C++ processes replace the
background “work” window.
• Read actions on stdin, write JSON event
stream to stdout.
• Main window spawns these processes, binds to
their stdio streams, and re-broadcasts events.
28. C++ tradeoffs
• In JavaScript, control flow is hard.
(Promises, async/await, etc.)
• In C++, control flow is easy but memory is hard.
(Sync I/O on a few threads.)
• The SQLite bindings for C are incredibly fast.
• Shipping C++ cross-platform is hard.
29. C++ is kind of… pretty?
• The C++11 syntax is really elegant.
• Modern features like condition_variable allow
the sync engine to do almost no polling.
32. Takeaways
• Keep your dependencies light, use built-in APIs
when possible. Check require tree with Devtron!
• Use “Paint Flashing” to identify crippling CSS /
layout issues. Minimize timer use if possible.
• Combining Electron and C++ (or Haskell, or Go,
etc.) isn’t that crazy.