Gaining Operational Insights out of Your Logs

•

2 likes•1,497 views

The document discusses gaining operational insights from logs by loading streaming log data into Amazon Elasticsearch Service. It describes common log sources on AWS including CloudFront access logs, S3 server access logs, ELB access logs, CloudTrail, and VPC flow logs. It then explains how to load streaming data from these sources into Amazon ES using AWS Lambda from Amazon S3, CloudWatch Logs, Kinesis Firehose, or Logstash. The presentation concludes with a demo of a real world scenario.

Technology

Gaining Operational Insights
out of your logs
Kobi Biton, Solutions Architect

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Agenda
• Common Log sources on AWS
• Loading Streaming data into Amazon
Elasticsearch Service
• Demo: Real world scenario

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
We want to turn this…
{
"eventTime": "2016-06-06T09:59:55Z",
"eventSource": "signin.amazonaws.com",
"eventName": "ConsoleLogin",
"awsRegion": "us-east-1",
"sourceIPAddress": ”169.254.169.254",
"userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0)
Gecko/20100101 Firefox/38.0",
"errorMessage": "Failed authentication",
"requestParameters": null,
"responseElements": {
"ConsoleLogin": "Failure"
},
"additionalEventData": {
"LoginTo": "SomeInfo"
"MFAUsed": "Yes"
},
{...}

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Into this…

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
And then optionally this…
Amazon
SNS
Topic
AWS
Lambda
Cloudwatch
Logs
Alarms

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Common Log Sources
• Amazon Cloudfront access logs , S3 server
access logs
• Elastic Load Balancer access logs
• AWS Cloudtrail , VPC Flow Logs
• Amazon SNS notifications
• Amazon EC2 OS Logs / Custom application
logs

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Loading Streaming
Data Into Amazon
Elasticsearch Service

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
From Amazon S3
Amazon ES
Domain
AWS
Lambda
ELB Logs
Cloudfront
Access Logs
S3 Access
Logs
S3

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS
Cloudtrail
VPC Flow
Logs
Cloudwatch
Logs
Amazon ES
Domain
AWS
Lambda
From Amazon Cloudwatch Logs
Cloudwatch Alarms

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Kinesis Firehose
Agent
Amazon
Kinesis
Firehose
Amazon ES
Domain
Via Amazon Kinesis Firehose

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Logstash
Agent
Log Objects
Amazon ES
Domain
Via Logstash Agent

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
DEMO

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Appendix

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Useful Links:
• https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-
aws-integrations.html
• https://github.com/awslabs/cloudwatch-logs-subscription-
consumer/tree/master/configuration/kibana
• https://docs.aws.amazon.com/firehose/latest/dev/writing-with-agents.html
• https://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/Su
bscriptions.html
• https://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/M
onitoringLogData.html

What's hot

Amazon Lightsail

Amazon Web Services

Scale, baby, scale!

Julien SIMON

Log analytics is a common big data use case that allows you to analyze log data from websites, mobile devices, servers, sensors, and more for a wide variety of applications including digital marketing, application monitoring, fraud detection, ad tech, gaming, and IoT. In this tech talk, we will walk you step-by-step through the process of building an end-to-end analytics solution that ingests, transforms, and loads streaming data using Amazon Kinesis Firehose, Amazon Kinesis Analytics and AWS Lambda. The processed data will be saved to an Amazon Elasticsearch Service cluster, and we will use Kibana to visualize the data in near real-time. Learning Objectives: 1. Reference architecture for building a complete log analytics solution 2. Overview of the services used and how they fit together 3. Best practices for log analytics implementation

Log Analytics with Amazon Elasticsearch Service and Amazon Kinesis - March 20...

Amazon Web Services

台灣客戶經驗分享: 零售品牌全通路經營-數位轉型新挑戰

Amazon Web Services

數位媒體雲端儲存案例和技術分享 (AWS Storage Options for Media Industry)

Amazon Web Services

AWS Lambda 與 Amazon API Gateway 新功能介紹

Amazon Web Services

Using Amazon CloudWatch Events, AWS Lambda and Spark Streaming to Process E...

Julien SIMON

Introduction to Amazon EC2 Spot Instances

Amazon Web Services

AWSome Day Dublin - June 2016

Amazon Web Services

Customers using AWS benefit from over 1,800 security and compliance controls built into the AWS platform and operations. In this session, you will learn how to take advantage of the advanced security features of the AWS platform to gain the visibility, agility, and control needed to be more secure in the cloud than in legacy environments. We'll take a look at several reference architectures for common workloads and highlight the innovative ways customers are using AWS to manage security more efficiently. After attending this session, you will be familiar with the shared security responsibility model and how you can inherit controls from the rich compliance and accreditation programs maintained by AWS.

SEC301 Security @ (Cloud) Scale

Amazon Web Services

GOTO Stockholm - AWS Lambda - Logic in the cloud without a back-end

Ian Massingham

AWS AWSome Day Roadshow Intro

Ian Massingham

Storage & Content Delivery

Amazon Web Services

PHP Office Hours

Amazon Web Services

Securing The AWS Cloud, Steve Riley, AWS Events, April 2010

Amazon Web Services

Whether you’re a cash-strapped startup or an enterprise optimizing spend, it pays to run cost-efficient architectures on AWS. This session reviews a wide range of cost planning, monitoring, and optimization strategies, featuring real-world experience from AWS customers. We’ll cover how you can effectively combine EC2 On-Demand, Reserved, and Spot instances to handle different use cases, leveraging auto scaling to match capacity to workload, choosing the most optimal instance type through load testing, taking advantage of multi-AZ support, and using CloudWatch to monitor usage and automatically shut off resources when not in use. We'll discuss taking advantage of tiered storage and caching, offloading content to Amazon CloudFront to reduce back-end load, and getting rid of your back end entirely, by leveraging AWS high-level services. We will also showcase simple tools to help track and manage costs, including the AWS Cost Explorer, Billing Alerts, and Trusted Advisor. This session will be your pocket guide for running cost effectively in the Amazon cloud. Watch the re:Invent recording here: https://www.youtube.com/watch?v=SG1DsYgeGEk

AWS Pop-up Loft Berlin: Cache is King - Running Lean Architectures: Optimizin...

AWS Germany

AWS Cloud Computing for Startups Werner Vogels -part i

Amazon Web Services

You want to launch your online platform and from a technical perspective you are wondering where to start and how to optimize your architecture? Cloud Computing presents several advantages such as scaling whenever you want your app our your Website. The hardest part is to define where to begin! During this 45 minutes workshop, Julien Simon will share with you the best practices to scale your platform from 0 to millions of users. He will present: - How to combine efficiently the tools Amazon Web Services provides, - How to set up the best architecture for your platform - How to scale your infrastructure in the Cloud. Before joining AWS, Julien worked as CTO of Viadeo and Aldebaran Robotics. He also spent more than 3 years as VP Engineering at Criteo. He is particularly interested by architecture, performance, deployment, scalability and data.

"How to optimize the architecture of your platform" by Julien Simon

TheFamily

支援大規模流量的網站應用程式雲端架構 (Web Applications on AWS)

Amazon Web Services

The Cloud as a Platform - Cloud Connections 2011 Keynote - Jinesh Varia

Amazon Web Services

What's hot (20)

Amazon Lightsail

Scale, baby, scale!

Log Analytics with Amazon Elasticsearch Service and Amazon Kinesis - March 20...

台灣客戶經驗分享: 零售品牌全通路經營-數位轉型新挑戰

數位媒體雲端儲存案例和技術分享 (AWS Storage Options for Media Industry)

AWS Lambda 與 Amazon API Gateway 新功能介紹

Using Amazon CloudWatch Events, AWS Lambda and Spark Streaming to Process E...

Introduction to Amazon EC2 Spot Instances

AWSome Day Dublin - June 2016

SEC301 Security @ (Cloud) Scale

GOTO Stockholm - AWS Lambda - Logic in the cloud without a back-end

AWS AWSome Day Roadshow Intro

Storage & Content Delivery

PHP Office Hours

Securing The AWS Cloud, Steve Riley, AWS Events, April 2010

AWS Pop-up Loft Berlin: Cache is King - Running Lean Architectures: Optimizin...

AWS Cloud Computing for Startups Werner Vogels -part i

"How to optimize the architecture of your platform" by Julien Simon

支援大規模流量的網站應用程式雲端架構 (Web Applications on AWS)

The Cloud as a Platform - Cloud Connections 2011 Keynote - Jinesh Varia

Viewers also liked

MongoDB Operational Best Practices (mongosf2012)

Scott Hernandez

Petty Cash Management - How To Manage Logs and Transactions

David Olson

Managing the logs of your (Rails) applications - RailsWayCon 2011

lennartkoopmann

"Grand Challenges" of Log Management

Anton Chuvakin

In the event of a disaster, you need to be able to recover lost data quickly to ensure business continuity. For critical applications, keeping your time to recover and data loss to a minimum as well as optimizing your overall capital expense can be challenging. This session presents AWS features and services along with Disaster Recovery architectures that you can leverage when building highly available and disaster resilient applications. We will provide recommendations on how to improve your Disaster Recovery plan and discuss example scenarios showing how to recover from a disaster.

Deploying a Disaster Recovery Site on AWS: Minimal Cost with Maximum Efficiency

Amazon Web Services

Getting Started with Amazon Aurora

Amazon Web Services

AWSome Day Cork | Technical Track

Amazon Web Services

Zombie Apocalypse Workshop by Warren Santer and Kyle Somers, Solutions Archit...

Amazon Web Services

Wild rydes serverless website workshop

Amazon Web Services

AWS Summit Sydney 2014 | Moving to the Cloud. What does it Mean to your Business

Amazon Web Services

AWS provides a platform that is ideally suited for deploying highly available and reliable systems that can scale with a minimal amount of human interaction. This talk describes a set of architectural patterns that support highly available services that are also scalable, low cost, low latency and allow for agile development practices. We walk through the various architectural decisions taken for each tier and explain our choices for appropriate AWS services and building blocks to ensure the security, scale, availability and reliability of the application.

AWS Summit Sydney 2014 | Running your First Application on AWS

Amazon Web Services

Media Content Ingest, Storage, and Archiving with AWS - John Downey, Amazon W...

Amazon Web Services

Customer Sharing: Weather Risk - Weather on the Cloud

Amazon Web Services

In this session you can learn through best practices when launching on AWS by making the platform work for you, leveraging automation tools and dedicated expertise during the development process, and making your provider accountable. In this session you can learn how UltraServe collaborates with AWS customers to compliment internal capabilities by sharing learnings from a collective of customers, taking accountability for elements which may not be core to your business, allowing you to leverage additional scale efficiencies.

AWS Summit Sydney 2014 | Reduce Risk in eCommerce Projects through Outcome Ba...

Amazon Web Services

Many companies recognize the use of data analytics as an opportunity to better understand their customers and gain a lead on their competition. The ability to get better insight from vast amounts of unstructured data, coming from a multitude of sources, can give businesses the advantage in an industry where even the smallest improvement can mean a big difference. Amazon Web Services offers a range of big data, analytics and storage solutions that are used by companies such as NASDAQ, Bankinter and S&P Capital to deliver a highly secure and agile platform. Join this session and learn how it allows customers to start on a small scale but grow as their business requires, giving them the agility they need to deliver cutting edge solutions to their customers without any upfront CAPEX investment.

AWS Big Data Analytics IP Expo 2013

Amazon Web Services

AWS Enterprise Summit London | AWS as an Agile Enabler at The Co-operative

Amazon Web Services

Intended for customers who have (or will have) thousands of instances on AWS, this session is about reducing the complexity of managing costs for these large fleets so they run efficiently. Attendees will learn about common roadblocks that prevent large customers from cost optimizing, tools they can use to efficiently remove those roadblocks, and techniques to monitor their rate of cost optimization. The session will include a case study that will talk in detail about the millions of dollars saved using these techniques. Customers will learn about a range of templates they can use to quickly implement these techniques, and also partners who can help them implement these templates.

Cost Optimization at Scale

Amazon Web Services

Argus media & amazon cloud search

Amazon Web Services

Customer Sharing: HTC - What is in AWS Cloud for me?

Amazon Web Services

AWS launched in 2006, and since then we have released more than 530 services, features, and major announcements. Every year, we outpace the previous year in launches and are continuously accelerating the pace of innovation across the organization. Ever wonder how we formulate customer-centric ideas, turn them into features and services, and get them to market quickly? This session dives deep into how an idea becomes a service at AWS and how we continue to evolve the service after release through innovation at every level. We even spill the beans on how we manage operational excellence across our services to ensure the highest possible availability. Come learn about the rapid pace of innovation at AWS, and the culture that formulates magic behind the scenes.

AWS Summit Auckland 2014 | Managing the Pace of Innovation: Behind the Scenes...

Amazon Web Services

Viewers also liked (20)

MongoDB Operational Best Practices (mongosf2012)

Petty Cash Management - How To Manage Logs and Transactions

Managing the logs of your (Rails) applications - RailsWayCon 2011

"Grand Challenges" of Log Management

Deploying a Disaster Recovery Site on AWS: Minimal Cost with Maximum Efficiency

Getting Started with Amazon Aurora

AWSome Day Cork | Technical Track

Zombie Apocalypse Workshop by Warren Santer and Kyle Somers, Solutions Archit...

Wild rydes serverless website workshop

AWS Summit Sydney 2014 | Moving to the Cloud. What does it Mean to your Business

AWS Summit Sydney 2014 | Running your First Application on AWS

Media Content Ingest, Storage, and Archiving with AWS - John Downey, Amazon W...

Customer Sharing: Weather Risk - Weather on the Cloud

AWS Summit Sydney 2014 | Reduce Risk in eCommerce Projects through Outcome Ba...

AWS Big Data Analytics IP Expo 2013

AWS Enterprise Summit London | AWS as an Agile Enabler at The Co-operative

Cost Optimization at Scale

Argus media & amazon cloud search

Customer Sharing: HTC - What is in AWS Cloud for me?

AWS Summit Auckland 2014 | Managing the Pace of Innovation: Behind the Scenes...

Similar to Gaining Operational Insights out of Your Logs

Serverless computing allows you to build and run applications without the need for provisioning or managing servers. With serverless computing, you can build web, mobile, and IoT backends; run stream processing or big data workloads; run chatbots, and more. In this session, you’ll learn how to get started with serverless computing with AWS Lambda, which lets you run code without provisioning or managing servers. We’ll introduce you to the basics of building with Lambda and how you can benefit from features such as continuous scaling, built-in high availability, integrations with AWS and third-party apps, and subsecond metering pricing. We’ll also introduce you to the broader portfolio of AWS services that help you build serverless applications with Lambda, including Amazon API Gateway, Amazon DynamoDB, AWS Step Functions, and more.

Getting Started with AWS Lambda and Serverless Computing

Amazon Web Services

Security oversight requires you to keep tabs on the near-real-time state of resources within your environment and provides challenges such as the management of large data sets from multiple sources. Join us to learn how you can build your own analysis and monitoring platform, providing both visual and audible alerting. Speaker: Michael Stringer, Solutions Architect, AWS & Sumit Patel, Solutions Architect, AWS

Visualise and Voice-Enable Your Security

Amazon Web Services

Going Serverless at AWS Startup Day Bangalore

Madhusudan Shekar

Do you want to ramp up your knowledge of AWS analytics services and launch your first big data application on the cloud? In this session, we walk you through simplifying big data processing as a data bus comprising of ingestion, storage, processing, and visualization. You build a big data application using AWS managed services, including Amazon Athena, Amazon Kinesis, Amazon EMR, AWS Glue, Amazon Redshift, Amazon QuickSight, and Amazon S3. Along the way, we review architecture design patterns for big data applications and give you access to a take-home lab so you can rebuild and customize the application yourself. To get the most from this session, bring your own laptop and have some familiarity with AWS services.

BDA309 Build Your First Big Data Application on AWS

Amazon Web Services

Bootcamp: Getting Started on AWS

Amazon Web Services

Visualise and Voice-Enable Your Security Security oversight requires you to keep tabs on the near-real-time state of resources within your environment and provides challenges such as the management of large data sets from multiple sources. This session will demonstrate how you can build your own analysis and monitoring platform, providing both visual and audible alerting. Michael Stringer, Solutions Architect, Amazon Web Services and Sumit Patel, Solutions Architect, Amazon Web Services

Visualise and Voice-Enable Your Security - AWS Summit Sydney 2018

Amazon Web Services

Serverless Development Deep Dive

Amazon Web Services

Nowadays, it’s common for a web server to be fronted by a global content delivery service, such as Amazon CloudFront, to accelerate delivery of websites, APIs, media content, and other web assets. Website administrators and developers want to generate insights in order to improve website availability through bot detection and mitigation, by optimizing web content based on the devices and browser used, by reducing perceived latency by caching a popular object closer to its viewer, and so on. In this session, we dive deep into building an end-to-end serverless analytics solution to analyze Amazon CloudFront access logs, both at rest and in transit, using Amazon Athena and Amazon Kinesis Analytics, respectively, and we generate visualization insights using Amazon QuickSight. Join a discussion with AWS solution architects to learn more about the various ways to generate insights to improve the overall perceived experience for your website users.

I Want to Analyze and Visualize Website Access Logs, but Why Do I Need Server...

Amazon Web Services

Want to ramp up your knowledge of AWS big data web services and launch your first big data application on the cloud? We walk you through simplifying big data processing as a data bus comprising ingest, store, process, and visualize. You build a big data application using AWS managed services, including Amazon Athena, Amazon Kinesis, Amazon DynamoDB, and Amazon S3. Along the way, we review architecture design patterns for big data applications and give you access to a take-home lab so that you can rebuild and customize the application yourself. You should bring your own laptop and have some familiarity with AWS services to get the most from this session.

ABD317_Building Your First Big Data Application on AWS - ABD317

Amazon Web Services

Getting started with streaming analytics: Deep Dive

javier ramirez

In this introductory chalk talk, we cover how to secure your resources in the cloud for common AWS workloads such as Amazon EC2 computing, database, and serverless. We cover security best practices recommended by AWS for each workload using simple and effective identity and networking techniques. Learn how and why these controls do what they do, and come away with the ability to interpret and apply AWS identity and network access controls.

Discuss How to Secure Your Virtual Data Center in the Cloud (NET210-R1) - AWS...

Amazon Web Services

Organizations need to gain insight and knowledge from a growing number of Internet of Things (IoT) APIs clickstreams comprised of unstructured and log data sources. However, organizations are often limited by legacy data warehouses and ETL processes that were designed for transactional data. In this session, we’ll introduce the key ETL features of AWS Glue through use cases ranging from scheduled nightly data warehouse loads to near real-time, event-driven ETL flows for your data lake. We’ll also discuss how to build scalable, efficient and serverless ETL pipelines using AWS Glue.

Build and run applications without thinking about servers

Amazon Web Services

Serverless Architectural Patterns - GOTO Amsterdam

Boaz Ziniman

In this session, we explore features and functions of AWS IoT services. The first part of the session covers AWS IoT fundamentals and our partner ecosystem followed by discussing about AWS IoT services in greater detail, review best practices for IoT solutions, and look at some common architectural patterns. With this foundation in place, explore a use case for IoT applications. Leave this session with an understanding of how to start building IoT applications with AWS IoT services.

IoT Building Blocks From Edge Devices to Analytics in the Cloud

Amazon Web Services

AWS IoT is a managed cloud platform that lets connected devices easily and securely interact with cloud applications and other devices. As an IoT developer, you will want to interact with AWS services like Kinesis, Lambda, and Amazon Machine Learning to get the most from your IoT application. In this session, we will do a deep dive on how to define rules in the Rules Engine, or retrieve the last known and desired state of device using Device Shadows, learn about the use cases and benefits of AWS Greengrass, and routing data from devices to AWS services to leverage the entire cloud for your Internet of Things application.

Deep Dive on the IoT at AWS

Amazon Web Services

Join us for this first-ever advanced design and best practices workshop, designed to demonstrate the breadth of AWS serverless offerings and how the components work together. In this interactive workshop, we review the evolution of an e-commerce company that starts with a low-effort serverless product catalog, scales to a million daily users, and then adds analytics and near real-time monitoring. As we progress through the workshop, we dive deeply into AWS serverless services, such as Amazon DynamoDB, AWS Lambda, and Amazon Kinesis. We also use Amazon S3, Amazon API Gateway, Amazon Cognito, and other services that enable you to optimize costs and improve performance. Basic knowledge of DynamoDB, Lambda, and Kinesis is required. Bring your laptop and power supply to this session.

Building Serverless Applications with Amazon DynamoDB & AWS Lambda - Workshop...

Amazon Web Services

This presentation will include a deep dive into the code behind multiple security automation and remediation functions. This session will consider potential use cases, as well as feature a demonstration of a proposed script, and then walk through the code set to explain the various challenges and solutions of the intended script. All examples of code will be previously unreleased and will feature integration with services such as Trusted Advisor and Macie. All code will be released as OSS after re:Invent.

Five New Security Automation Improvements You Can Make by Using Amazon CloudW...

Amazon Web Services

An Active Case Study on Insider Threat Detection in your Applications

Amazon Web Services

Migrating Microsoft Applications to AWS like an Expert

Injae Kwak

Notice: This Workshop requires a laptop computer and an active AWS account with Administrator privileges. This workshop gives you an opportunity to develop a solution that can continuously monitor for and detect a realistic threat by analyzing AWS CloudTrail log data. Participants are provided with a CloudTrail data source and some clues to get started. Then you have to design a system that can process the logs, detect the threat, and trigger an alarm. You can make use of any AWS services that can assist in this endeavor, such as AWS Lambda for serverless detection logic, Amazon CloudWatch or Amazon SNS for alarming and notification, Amazon S3 for data and configuration storage, and more.

Using AWS CloudTrail Logs for Scalable, Automated Anomaly Detection - SID341 ...

Amazon Web Services

Similar to Gaining Operational Insights out of Your Logs (20)

Getting Started with AWS Lambda and Serverless Computing

Visualise and Voice-Enable Your Security

Going Serverless at AWS Startup Day Bangalore

BDA309 Build Your First Big Data Application on AWS

Bootcamp: Getting Started on AWS

Visualise and Voice-Enable Your Security - AWS Summit Sydney 2018

Serverless Development Deep Dive

I Want to Analyze and Visualize Website Access Logs, but Why Do I Need Server...

ABD317_Building Your First Big Data Application on AWS - ABD317

Getting started with streaming analytics: Deep Dive

Discuss How to Secure Your Virtual Data Center in the Cloud (NET210-R1) - AWS...

Build and run applications without thinking about servers

Serverless Architectural Patterns - GOTO Amsterdam

IoT Building Blocks From Edge Devices to Analytics in the Cloud

Deep Dive on the IoT at AWS

Building Serverless Applications with Amazon DynamoDB & AWS Lambda - Workshop...

Five New Security Automation Improvements You Can Make by Using Amazon CloudW...

An Active Case Study on Insider Threat Detection in your Applications

Migrating Microsoft Applications to AWS like an Expert

Using AWS CloudTrail Logs for Scalable, Automated Anomaly Detection - SID341 ...

More from Amazon Web Services

Il Forecasting è un processo importante per tantissime aziende e viene utilizzato in vari ambiti per cercare di prevedere in modo accurato la crescita e distribuzione di un prodotto, l’utilizzo delle risorse necessarie nelle linee produttive, presentazioni finanziarie e tanto altro. Amazon utilizza delle tecniche avanzate di forecasting, in parte questi servizi sono stati messi a disposizione di tutti i clienti AWS. In questa sessione illustreremo come pre-processare i dati che contengono una componente temporale e successivamente utilizzare un algoritmo che a partire dal tipo di dato analizzato produce un forecasting accurato.

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...

Amazon Web Services

La varietà e la quantità di dati che si crea ogni giorno accelera sempre più velocemente e rappresenta una opportunità irripetibile per innovare e creare nuove startup. Tuttavia gestire grandi quantità di dati può apparire complesso: creare cluster Big Data su larga scala sembra essere un investimento accessibile solo ad aziende consolidate. Ma l’elasticità del Cloud e, in particolare, i servizi Serverless ci permettono di rompere questi limiti. Vediamo quindi come è possibile sviluppare applicazioni Big Data rapidamente, senza preoccuparci dell’infrastruttura, ma dedicando tutte le risorse allo sviluppo delle nostre le nostre idee per creare prodotti innovativi.

Big Data per le Startup: come creare applicazioni Big Data in modalità Server...

Amazon Web Services

Ora puoi utilizzare Amazon Elastic Kubernetes Service (EKS) per eseguire pod Kubernetes su AWS Fargate, il motore di elaborazione serverless creato per container su AWS. Questo rende più semplice che mai costruire ed eseguire le tue applicazioni Kubernetes nel cloud AWS.In questa sessione presenteremo le caratteristiche principali del servizio e come distribuire la tua applicazione in pochi passaggi

Esegui pod serverless con Amazon EKS e AWS Fargate

Amazon Web Services

Vent'anni fa Amazon ha attraversato una trasformazione radicale con l'obiettivo di aumentare il ritmo dell'innovazione. In questo periodo abbiamo imparato come cambiare il nostro approccio allo sviluppo delle applicazioni ci ha permesso di aumentare notevolmente l'agilità, la velocità di rilascio e, in definitiva, ci ha consentito di creare applicazioni più affidabili e scalabili. In questa sessione illustreremo come definiamo le applicazioni moderne e come la creazione di app moderne influisce non solo sull'architettura dell'applicazione, ma sulla struttura organizzativa, sulle pipeline di rilascio dello sviluppo e persino sul modello operativo. Descriveremo anche approcci comuni alla modernizzazione, compreso l'approccio utilizzato dalla stessa Amazon.com.

Costruire Applicazioni Moderne con AWS

Amazon Web Services

L’utilizzo dei container è in continua crescita. Se correttamente disegnate, le applicazioni basate su Container sono molto spesso stateless e flessibili. I servizi AWS ECS, EKS e Kubernetes su EC2 possono sfruttare le istanze Spot, portando ad un risparmio medio del 70% rispetto alle istanze On Demand. In questa sessione scopriremo insieme quali sono le caratteristiche delle istanze Spot e come possono essere utilizzate facilmente su AWS. Impareremo inoltre come Spreaker sfrutta le istanze spot per eseguire applicazioni di diverso tipo, in produzione, ad una frazione del costo on-demand!

Come spendere fino al 90% in meno con i container e le istanze spot

Amazon Web Services

In recent months, many customers have been asking us the question – how to monetise Open APIs, simplify Fintech integrations and accelerate adoption of various Open Banking business models. Therefore, AWS and FinConecta would like to invite you to Open Finance marketplace presentation on October 20th. Event Agenda : Open banking so far (short recap) • PSD2, OB UK, OB Australia, OB LATAM, OB Israel Intro to Open Finance marketplace • Scope • Features • Tech overview and Demo The role of the Cloud The Future of APIs • Complying with regulation • Monetizing data / APIs • Business models • Time to market One platform for all: a Strategic approach Q&A

Open banking as a service

Amazon Web Services

Per creare valore e costruire una propria offerta differenziante e riconoscibile, le startup di successo sanno come combinare tecnologie consolidate con componenti innovativi creati ad hoc. AWS fornisce servizi pronti all'utilizzo e, allo stesso tempo, permette di personalizzare e creare gli elementi differenzianti della propria offerta. Concentrandoci sulle tecnologie di Machine Learning, vedremo come selezionare i servizi di intelligenza artificiale offerti da AWS e, anche attraverso una demo, come costruire modelli di Machine Learning personalizzati utilizzando SageMaker Studio.

Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...

Amazon Web Services

Con l'approccio tradizionale al mondo IT per molti anni è stato difficile implementare tecniche di DevOps, che finora spesso hanno previsto attività manuali portando di tanto in tanto a dei downtime degli applicativi interrompendo l'operatività dell'utente. Con l'avvento del cloud, le tecniche di DevOps sono ormai a portata di tutti a basso costo per qualsiasi genere di workload, garantendo maggiore affidabilità del sistema e risultando in dei significativi miglioramenti della business continuity. AWS mette a disposizione AWS OpsWork come strumento di Configuration Management che mira ad automatizzare e semplificare la gestione e i deployment delle istanze EC2 per mezzo di workload Chef e Puppet. Scopri come sfruttare AWS OpsWork a garanzia e affidabilità del tuo applicativo installato su Instanze EC2.

OpsWorks Configuration Management: automatizza la gestione e i deployment del...

Amazon Web Services

Vuoi conoscere le opzioni per eseguire Microsoft Active Directory su AWS? Quando si spostano carichi di lavoro Microsoft in AWS, è importante considerare come distribuire Microsoft Active Directory per supportare la gestione, l'autenticazione e l'autorizzazione dei criteri di gruppo. In questa sessione, discuteremo le opzioni per la distribuzione di Microsoft Active Directory su AWS, incluso AWS Directory Service per Microsoft Active Directory e la distribuzione di Active Directory su Windows su Amazon Elastic Compute Cloud (Amazon EC2). Trattiamo argomenti quali l'integrazione del tuo ambiente Microsoft Active Directory locale nel cloud e l'utilizzo di applicazioni SaaS, come Office 365, con AWS Single Sign-On.

Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads

Amazon Web Services

Computer Vision con AWS

Amazon Web Services

Amazon Web Services e VMware organizzano un evento virtuale gratuito il prossimo mercoledì 14 Ottobre dalle 12:00 alle 13:00 dedicato a VMware Cloud ™ on AWS, il servizio on demand che consente di eseguire applicazioni in ambienti cloud basati su VMware vSphere® e di accedere ad una vasta gamma di servizi AWS, sfruttando a pieno le potenzialità del cloud AWS e tutelando gli investimenti VMware esistenti. Molte organizzazioni sfruttano i vantaggi del cloud migrando i propri carichi di lavoro Oracle e assicurandosi notevoli vantaggi in termini di agilità ed efficienza dei costi. La migrazione di questi carichi di lavoro, può creare complessità durante la modernizzazione e il refactoring delle applicazioni e a questo si possono aggiungere rischi di prestazione che possono essere introdotti quando si spostano le applicazioni dai data center locali.

Database Oracle e VMware Cloud on AWS i miti da sfatare

Amazon Web Services

Molte aziende oggi, costruiscono applicazioni con funzionalità di tipo ledger ad esempio per verificare lo storico di accrediti o addebiti nelle transazioni bancarie o ancora per tenere traccia del flusso supply chain dei propri prodotti. Alla base di queste soluzioni ci sono i database ledger che permettono di avere un log delle transazioni trasparente, immutabile e crittograficamente verificabile, ma sono strumenti complessi e onerosi da gestire. Amazon QLDB elimina la necessità di costruire sistemi personalizzati e complessi fornendo un database ledger serverless completamente gestito. In questa sessione scopriremo come realizzare un'applicazione serverless completa che utilizzi le funzionalità di QLDB.

Crea la tua prima serverless ledger-based app con QLDB e NodeJS

Amazon Web Services

Con l’ascesa delle architetture di microservizi e delle ricche applicazioni mobili e Web, le API sono più importanti che mai per offrire agli utenti finali una user experience eccezionale. In questa sessione impareremo come affrontare le moderne sfide di progettazione delle API con GraphQL, un linguaggio di query API open source utilizzato da Facebook, Amazon e altro e come utilizzare AWS AppSync, un servizio GraphQL serverless gestito su AWS. Approfondiremo diversi scenari, comprendendo come AppSync può aiutare a risolvere questi casi d’uso creando API moderne con funzionalità di aggiornamento dati in tempo reale e offline. Inoltre, impareremo come Sky Italia utilizza AWS AppSync per fornire aggiornamenti sportivi in tempo reale agli utenti del proprio portale web.

API moderne real-time per applicazioni mobili e web

Amazon Web Services

Molte organizzazioni sfruttano i vantaggi del cloud migrando i propri carichi di lavoro Oracle e assicurandosi notevoli vantaggi in termini di agilità ed efficienza dei costi. La migrazione di questi carichi di lavoro, può creare complessità durante la modernizzazione e il refactoring delle applicazioni e a questo si possono aggiungere rischi di prestazione che possono essere introdotti quando si spostano le applicazioni dai data center locali. In queste slide, gli esperti AWS e VMware presentano semplici e pratici accorgimenti per facilitare e semplificare la migrazione dei carichi di lavoro Oracle accelerando la trasformazione verso il cloud, approfondiranno l’architettura e dimostreranno come sfruttare a pieno le potenzialità di VMware Cloud ™ on AWS.

Database Oracle e VMware Cloud™ on AWS: i miti da sfatare

Amazon Web Services

Tools for building your MVP on AWS

Amazon Web Services

How to Build a Winning Pitch Deck

Amazon Web Services

Building a web application without servers

Amazon Web Services

Fundraising Essentials

Amazon Web Services

AWS_HK_StartupDay_Building Interactive websites while automating for efficien...

Amazon Web Services

Amazon Elastic Container Service (Amazon ECS) è un servizio di gestione dei container altamente scalabile, che semplifica la gestione dei contenitori Docker attraverso un layer di orchestrazione per il controllo del deployment e del relativo lifecycle. In questa sessione presenteremo le principali caratteristiche del servizio, le architetture di riferimento per i differenti carichi di lavoro e i semplici passi necessari per poter velocemente migrare uno o più dei tuo container.

Introduzione a Amazon Elastic Container Service

Amazon Web Services

More from Amazon Web Services (20)

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...

Big Data per le Startup: come creare applicazioni Big Data in modalità Server...

Esegui pod serverless con Amazon EKS e AWS Fargate

Costruire Applicazioni Moderne con AWS

Come spendere fino al 90% in meno con i container e le istanze spot

Open banking as a service

Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...

OpsWorks Configuration Management: automatizza la gestione e i deployment del...

Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads

Computer Vision con AWS

Database Oracle e VMware Cloud on AWS i miti da sfatare

Crea la tua prima serverless ledger-based app con QLDB e NodeJS

API moderne real-time per applicazioni mobili e web

Database Oracle e VMware Cloud™ on AWS: i miti da sfatare

Tools for building your MVP on AWS

How to Build a Winning Pitch Deck

Building a web application without servers

Fundraising Essentials

AWS_HK_StartupDay_Building Interactive websites while automating for efficien...

Introduzione a Amazon Elastic Container Service

Recently uploaded

EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER

MadyBayot

Artificial Intelligence Chap.5 : Uncertainty

Khushali Kathiriya

GenAI Risks & Security Meetup 01052024.pdf

lior mazor

MySQL Webinar, presented on the 25th of April, 2024. Summary: MySQL solutions enable the deployment of diverse Database Architectures tailored to specific needs, including High Availability, Disaster Recovery, and Read Scale-Out. With MySQL Shell's AdminAPI, administrators can seamlessly set up, manage, and monitor these solutions, ensuring efficiency and ease of use in their administration. MySQL Router, on the other hand, provides transparent routing from the application traffic to the backend servers in the architectures, requiring minimal configuration. Completely built in-house and supported by Oracle, these solutions have been adopted by enterprises of all sizes for their business-critical applications. In this presentation, we'll delve into various database architecture solutions to help you choose the right one based on your business requirements. Focusing on technical details and the latest features to maximize the potential of these solutions.

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...

Miguel Araújo

Modernizing Securities Finance: The cloud-native prime brokerage platform transforming capital markets. Madhu Subbu, Managing Director, Head of Securities Finance Engineering Apidays Singapore 2024: Connecting Customers, Business and Technology (April 17 & 18, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu

apidays

AWS Community Day CPH - Three problems of Terraform

Andrey Devyatkin

How to Troubleshoot Apps for the Modern Connected Worker

ThousandEyes

Accelerating FinTech Innovation: Unleashing API Economy and GenAI Vasa Krishnan, Chief Technology Officer - FinResults Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...

apidays

2024: Domino Containers - The Next Step. News from the Domino Container commu...

Martijn de Jong

AXA XL - Insurer Innovation Award Americas 2024

The Digital Insurer

Abhishek Deb(1), Mr Abdul Kalam(2) M. Des (UX) , School of Design, DIT University , Dehradun. This paper explores the future potential of AI-enabled smartphone processors, aiming to investigate the advancements, capabilities, and implications of integrating artificial intelligence (AI) into smartphone technology. The research study goals consist of evaluating the development of AI in mobile phone processors, analyzing the existing state as well as abilities of AI-enabled cpus determining future patterns as well as chances together with reviewing obstacles as well as factors to consider for more growth.

Exploring the Future Potential of AI-Enabled Smartphone Processors

debabhi2

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving

Edi Saputra

This presentations targets students or working professionals. You may know Google for search, YouTube, Android, Chrome, and Gmail, but did you know Google has many developer tools, platforms & APIs? This comprehensive yet still high-level overview outlines the most impactful tools for where to run your code, store & analyze your data. It will also inspire you as to what's possible. This talk is 50 minutes in length.

Powerful Google developer tools for immediate impact! (2023-24 C)

wesley chun

Scaling API-first – The story of a global engineering organization Ian Reasor, Senior Computer Scientist - Adobe Radu Cotescu, Senior Computer Scientist - Adobe Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

apidays

Architecting Cloud Native Applications

WSO2

Automating Google Workspace (GWS) & more with Apps Script

wesley chun

A Beginners Guide to Building a RAG App Using Open Source Milvus

Zilliz

The Good, the Bad and the Governed - Why is governance a dirty word? David O'Neill, Chief Operating Officer - APIContext Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

apidays

Data Cloud, More than a CDP by Matt Robison

Anna Loughnan Colquhoun

Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows. We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases. This video focuses on the deployment of external web forms using Jotform for Bonterra Impact Management. This solution can be customized to your organization’s needs and deployed to support the common use cases below: - Intake and consent - Assessments - Surveys - Applications - Program registration Interested in deploying web form automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.

Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...

Jeffrey Haguewood

Recently uploaded (20)

EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER

Artificial Intelligence Chap.5 : Uncertainty

GenAI Risks & Security Meetup 01052024.pdf

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...

Apidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu

AWS Community Day CPH - Three problems of Terraform

How to Troubleshoot Apps for the Modern Connected Worker

Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...

2024: Domino Containers - The Next Step. News from the Domino Container commu...

AXA XL - Insurer Innovation Award Americas 2024

Exploring the Future Potential of AI-Enabled Smartphone Processors

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving

Powerful Google developer tools for immediate impact! (2023-24 C)

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

Architecting Cloud Native Applications

Automating Google Workspace (GWS) & more with Apps Script

A Beginners Guide to Building a RAG App Using Open Source Milvus

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

Data Cloud, More than a CDP by Matt Robison

Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...

Gaining Operational Insights out of Your Logs

1. Gaining Operational Insights out of your logs Kobi Biton, Solutions Architect

3. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. We want to turn this… { "eventTime": "2016-06-06T09:59:55Z", "eventSource": "signin.amazonaws.com", "eventName": "ConsoleLogin", "awsRegion": "us-east-1", "sourceIPAddress": ”169.254.169.254", "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0) Gecko/20100101 Firefox/38.0", "errorMessage": "Failed authentication", "requestParameters": null, "responseElements": { "ConsoleLogin": "Failure" }, "additionalEventData": { "LoginTo": "SomeInfo" "MFAUsed": "Yes" }, {...}

6. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Common Log Sources • Amazon Cloudfront access logs , S3 server access logs • Elastic Load Balancer access logs • AWS Cloudtrail , VPC Flow Logs • Amazon SNS notifications • Amazon EC2 OS Logs / Custom application logs

13.

14. kobibito@amazon.com

16. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Useful Links: • https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es- aws-integrations.html • https://github.com/awslabs/cloudwatch-logs-subscription- consumer/tree/master/configuration/kibana • https://docs.aws.amazon.com/firehose/latest/dev/writing-with-agents.html • https://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/Su bscriptions.html • https://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/M onitoringLogData.html

Gaining Operational Insights out of Your Logs

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to Gaining Operational Insights out of Your Logs

Similar to Gaining Operational Insights out of Your Logs (20)

More from Amazon Web Services

More from Amazon Web Services (20)

Recently uploaded

Recently uploaded (20)

Gaining Operational Insights out of Your Logs