This document discusses Big Data concepts and tools including Hadoop, HDFS, MapReduce, Pig Latin, Hive, and analyzing JSON data with Hive. It provides examples of using MapReduce with JavaScript and C# to count words in text data. It also summarizes HDFS, showing how it provides fault tolerance and handles user requests through replication across data nodes. Finally, it discusses analyzing Twitter data stored in JSON format using Hive to extract and interpret fields.
Berlin Open 09: Architekturen sozialer Netzwerke -- Fallbeispiel MySQLKaj Arnö
Netzwerke, wie LinkedIn, Twitter, Facebook, Xing, aber auch Flickr, Picasa, WordPress, YouTube u.v.a., steigen ein als neue Mittel für Marketing und Community Building. Dies betrifft Unternehmen, FOSS-Projekte sowie einzelne Akteure -- Entwickler, Forscher, Investoren. Dieser praxisbezogene Vortrag beschreibt die Möglichkeiten der Web 2.0 für eine effizientere Kommunikation, mit MySQL als Fallbeispiel.
Vom Bedürfnis zum Produkt - Integration von UX Methoden in der Softwareentwic...SIC! Software GmbH
Ziel des Projekts D4X ist die Entwicklung und der Einsatz von Methoden und Instrumenten, mit denen kleine und mittelständische Unternehmen (KMU) ihre Software attraktiver gestalten und zu einem positiven Erlebnis für Nutzer machen können.
Big Data - part 5/7 of "7 modern trends that every IT Pro should know about"Ibrahim Muhammadi
Presented by Ibrahim Muhammadi. Founder - AppWorx.cc
Big Data is revolutionizing how businesses make decisions now. More and more decisions and strategies are now based on data.
Berlin Open 09: Architekturen sozialer Netzwerke -- Fallbeispiel MySQLKaj Arnö
Netzwerke, wie LinkedIn, Twitter, Facebook, Xing, aber auch Flickr, Picasa, WordPress, YouTube u.v.a., steigen ein als neue Mittel für Marketing und Community Building. Dies betrifft Unternehmen, FOSS-Projekte sowie einzelne Akteure -- Entwickler, Forscher, Investoren. Dieser praxisbezogene Vortrag beschreibt die Möglichkeiten der Web 2.0 für eine effizientere Kommunikation, mit MySQL als Fallbeispiel.
Vom Bedürfnis zum Produkt - Integration von UX Methoden in der Softwareentwic...SIC! Software GmbH
Ziel des Projekts D4X ist die Entwicklung und der Einsatz von Methoden und Instrumenten, mit denen kleine und mittelständische Unternehmen (KMU) ihre Software attraktiver gestalten und zu einem positiven Erlebnis für Nutzer machen können.
Big Data - part 5/7 of "7 modern trends that every IT Pro should know about"Ibrahim Muhammadi
Presented by Ibrahim Muhammadi. Founder - AppWorx.cc
Big Data is revolutionizing how businesses make decisions now. More and more decisions and strategies are now based on data.
Big Data Applications & Analytics Motivation: Big Data and the Cloud; Centerp...Geoffrey Fox
Motivating Introduction to MOOC on Big Data from an applications point of view https://bigdatacoursespring2014.appspot.com/course
Course says:
Geoffrey motivates the study of X-informatics by describing data science and clouds. He starts with striking examples of the data deluge with examples from research, business and the consumer. The growing number of jobs in data science is highlighted. He describes industry trend in both clouds and big data.
He introduces the cloud computing model developed at amazing speed by industry. The 4 paradigms of scientific research are described with growing importance of data oriented version. He covers 3 major X-informatics areas: Physics, e-Commerce and Web Search followed by a broad discussion of cloud applications. Parallel computing in general and particular features of MapReduce are described. He comments on a data science education and the benefits of using MOOC's.
Big Data Applications & Analytics Motivation: Big Data and the Cloud; Center...Geoffrey Fox
Motivating Introduction to MOOC on Big Data from an applications point of view https://bigdatacoursespring2014.appspot.com/course
Course says:
Geoffrey motivates the study of X-informatics by describing data science and clouds. He starts with striking examples of the data deluge with examples from research, business and the consumer. The growing number of jobs in data science is highlighted. He describes industry trend in both clouds and big data.
He introduces the cloud computing model developed at amazing speed by industry. The 4 paradigms of scientific research are described with growing importance of data oriented version. He covers 3 major X-informatics areas: Physics, e-Commerce and Web Search followed by a broad discussion of cloud applications. Parallel computing in general and particular features of MapReduce are described. He comments on a data science education and the benefits of using MOOC's.
Azureday 2020 - The Edge talks - long road into the CloudRafal Warzycha
Edge Computing is gaining incredible hype and traction all over IoT community. Day by day we can see corporations and startups dive in this movement on both hardware and software level. Join me in the journey across the Cloud and beyond to the Edge. We will be climbing the peaks of real use cases to see the story behind all the buzzwords and fluff.Let’s analyze this based on real life case studies and demos of Edge Computing in practice.
Big data, open data and telepathy: technologies for smart, human-scale cities...Rick Robinson
How will cities and communities be successful in the future? Why will people want to live in them and what challenges will they face? Technologies such as big data, 3D printing, the internet of things and social media will be crucial enablers of resilient, vibrant and equitable cities and communities in the future; and technologies invented in coming years will quickly create possibilities that are hard for us to imagine today. But applying them successfully to create better places to live and do business is a challenge for personal and community leadership and business innovation, not just engineering.
20th Athens Big Data Meetup - 2nd Talk - Druid: under the coversAthens Big Data
Title: Druid: under the covers
Speaker: Peter Marshall (https://linkedin.com/in/amillionbytes/)
Date: Tuesday, January 28, 2020
Event: https://meetup.com/Athens-Big-Data/events/266900242/
Francis da costa rethinks the internet of things zd_netMeshDynamics
https://www.zdnet.com/article/francis-dacosta-rethinks-the-internet-of-things/ and see
https://thefrugalnetworker.files.wordpress.com/2014/05/francis_dacosta_rethinking_the_internet_of_things.mp3
This presentation as made as a tutorial at NCVPRIPG (http://www.iitj.ac.in/ncvpripg/) at IIT Jodhpur on 18-Dec-2013.
Kinect is a multimedia sensor from Microsoft. It is shipped as the touch-free console for Xbox 360 video gaming platform. Kinect comprises an RGB Camera, a Depth Sensor (IR Emitter and Camera) and a Microphone Array. It produces a multi-stream video containing RGB, depth, skeleton, and audio streams.
Compared to common depth cameras (laser or Time-of-Flight), the cost of a Kinect is quite low as it uses a novel structured light diffraction and triangulation technology to estimate the depth. In addition, Kinect is equipped with special software to detect human figures and to produce its 20-joints skeletons.
Though Kinect was built for touch-free gaming, its cost effectiveness and human tracking features have proved useful in many indoor applications beyond gaming like robot navigation, surveillance, medical assistance and animation.
A novel programmable attenuator based low Gm-OTA for biomedical applicationsHoopeer Hoopeer
dokumen, Scribd, SlideShare, book: Microelectronic-Devices-and-Circuits, Bit-Vector Pattern Matching Systems on the Basis of Analog-Digital Field Reprogrammable Arrays, Linear System Theory: The State Space Approach, National Academies Press (NAP), Chopper-Stabilized Low-Noise Multipath Operational Amplifier with Dual Ripple Rejection Loops, A Single Slope ADC With Row-Wise Noise Reduction Technique for CMOS Image Sensor, Docsity, TSpace, iThenticate, EBSCO, OpenAIRE, DOAJ, Novel Schmitt trigger and square-wave generator using single current amplifier, Integrated Systems Laboratory, xDevs, A Noninvasive Glucose Monitoring SoC Based on Single Wavelength Photoplethysmography, Slewing Mitigation Technique for Switched Capacitor Circuits, mathjax, Tezzaron Semiconductor, Continuous-Time ΔΣ Modulator, AMiner, A novel programmable attenuator based low Gm-OTA for biomedical applications
_Link24
Wer heutzutage in Big Data Projekten involviert ist, kommt an Hadoop-Tools, wie beispielsweise Hive, Spark oder Storm kaum noch vorbei.
Doch wie sieht es mit den Microsoft Bordmittel, genauer gesagt Visual Studio, C# und SQL, aus? Dieser Frage geht Sascha Dittmann in dieser Session nach.
Er zeigt anhand von Codebeispielen, und den Azure Data Lake Diensten, wie eine Umsetzung auch mit reinen Microsoft Tools möglich ist.
Hochskalierbare, relationale Datenbanken in Microsoft AzureSascha Dittmann
Die meisten Softwareprojekte fangen mit einer einfachen relationalen Datenbank an. Doch was soll man machen, wenn die Anforderungen an die Datenbank wachsen? Was, wenn die Datenbankgröße und die Anzahl der gleichzeitigen Abfragen stetig wächst? Was, wenn aus einzelnen Datenbanken ein Data Warehouse entsteht? Sascha Dittmann zeigt in dieser Session wie solche und ähnliche Szenarien mit Microsoft Azure umsetzbar sind. Er stellt dabei Technologien wie Azure SQL Database, Azure Elastic Pools und Azure SQL Data Warehouse im Detail vor und gibt Beispiele, sowie Tipps & Tricks aus der Praxis.
More Related Content
Similar to dotnet Cologne 2013 - Microsoft HD Insight für .NET Entwickler
Big Data Applications & Analytics Motivation: Big Data and the Cloud; Centerp...Geoffrey Fox
Motivating Introduction to MOOC on Big Data from an applications point of view https://bigdatacoursespring2014.appspot.com/course
Course says:
Geoffrey motivates the study of X-informatics by describing data science and clouds. He starts with striking examples of the data deluge with examples from research, business and the consumer. The growing number of jobs in data science is highlighted. He describes industry trend in both clouds and big data.
He introduces the cloud computing model developed at amazing speed by industry. The 4 paradigms of scientific research are described with growing importance of data oriented version. He covers 3 major X-informatics areas: Physics, e-Commerce and Web Search followed by a broad discussion of cloud applications. Parallel computing in general and particular features of MapReduce are described. He comments on a data science education and the benefits of using MOOC's.
Big Data Applications & Analytics Motivation: Big Data and the Cloud; Center...Geoffrey Fox
Motivating Introduction to MOOC on Big Data from an applications point of view https://bigdatacoursespring2014.appspot.com/course
Course says:
Geoffrey motivates the study of X-informatics by describing data science and clouds. He starts with striking examples of the data deluge with examples from research, business and the consumer. The growing number of jobs in data science is highlighted. He describes industry trend in both clouds and big data.
He introduces the cloud computing model developed at amazing speed by industry. The 4 paradigms of scientific research are described with growing importance of data oriented version. He covers 3 major X-informatics areas: Physics, e-Commerce and Web Search followed by a broad discussion of cloud applications. Parallel computing in general and particular features of MapReduce are described. He comments on a data science education and the benefits of using MOOC's.
Azureday 2020 - The Edge talks - long road into the CloudRafal Warzycha
Edge Computing is gaining incredible hype and traction all over IoT community. Day by day we can see corporations and startups dive in this movement on both hardware and software level. Join me in the journey across the Cloud and beyond to the Edge. We will be climbing the peaks of real use cases to see the story behind all the buzzwords and fluff.Let’s analyze this based on real life case studies and demos of Edge Computing in practice.
Big data, open data and telepathy: technologies for smart, human-scale cities...Rick Robinson
How will cities and communities be successful in the future? Why will people want to live in them and what challenges will they face? Technologies such as big data, 3D printing, the internet of things and social media will be crucial enablers of resilient, vibrant and equitable cities and communities in the future; and technologies invented in coming years will quickly create possibilities that are hard for us to imagine today. But applying them successfully to create better places to live and do business is a challenge for personal and community leadership and business innovation, not just engineering.
20th Athens Big Data Meetup - 2nd Talk - Druid: under the coversAthens Big Data
Title: Druid: under the covers
Speaker: Peter Marshall (https://linkedin.com/in/amillionbytes/)
Date: Tuesday, January 28, 2020
Event: https://meetup.com/Athens-Big-Data/events/266900242/
Francis da costa rethinks the internet of things zd_netMeshDynamics
https://www.zdnet.com/article/francis-dacosta-rethinks-the-internet-of-things/ and see
https://thefrugalnetworker.files.wordpress.com/2014/05/francis_dacosta_rethinking_the_internet_of_things.mp3
This presentation as made as a tutorial at NCVPRIPG (http://www.iitj.ac.in/ncvpripg/) at IIT Jodhpur on 18-Dec-2013.
Kinect is a multimedia sensor from Microsoft. It is shipped as the touch-free console for Xbox 360 video gaming platform. Kinect comprises an RGB Camera, a Depth Sensor (IR Emitter and Camera) and a Microphone Array. It produces a multi-stream video containing RGB, depth, skeleton, and audio streams.
Compared to common depth cameras (laser or Time-of-Flight), the cost of a Kinect is quite low as it uses a novel structured light diffraction and triangulation technology to estimate the depth. In addition, Kinect is equipped with special software to detect human figures and to produce its 20-joints skeletons.
Though Kinect was built for touch-free gaming, its cost effectiveness and human tracking features have proved useful in many indoor applications beyond gaming like robot navigation, surveillance, medical assistance and animation.
A novel programmable attenuator based low Gm-OTA for biomedical applicationsHoopeer Hoopeer
dokumen, Scribd, SlideShare, book: Microelectronic-Devices-and-Circuits, Bit-Vector Pattern Matching Systems on the Basis of Analog-Digital Field Reprogrammable Arrays, Linear System Theory: The State Space Approach, National Academies Press (NAP), Chopper-Stabilized Low-Noise Multipath Operational Amplifier with Dual Ripple Rejection Loops, A Single Slope ADC With Row-Wise Noise Reduction Technique for CMOS Image Sensor, Docsity, TSpace, iThenticate, EBSCO, OpenAIRE, DOAJ, Novel Schmitt trigger and square-wave generator using single current amplifier, Integrated Systems Laboratory, xDevs, A Noninvasive Glucose Monitoring SoC Based on Single Wavelength Photoplethysmography, Slewing Mitigation Technique for Switched Capacitor Circuits, mathjax, Tezzaron Semiconductor, Continuous-Time ΔΣ Modulator, AMiner, A novel programmable attenuator based low Gm-OTA for biomedical applications
_Link24
Wer heutzutage in Big Data Projekten involviert ist, kommt an Hadoop-Tools, wie beispielsweise Hive, Spark oder Storm kaum noch vorbei.
Doch wie sieht es mit den Microsoft Bordmittel, genauer gesagt Visual Studio, C# und SQL, aus? Dieser Frage geht Sascha Dittmann in dieser Session nach.
Er zeigt anhand von Codebeispielen, und den Azure Data Lake Diensten, wie eine Umsetzung auch mit reinen Microsoft Tools möglich ist.
Hochskalierbare, relationale Datenbanken in Microsoft AzureSascha Dittmann
Die meisten Softwareprojekte fangen mit einer einfachen relationalen Datenbank an. Doch was soll man machen, wenn die Anforderungen an die Datenbank wachsen? Was, wenn die Datenbankgröße und die Anzahl der gleichzeitigen Abfragen stetig wächst? Was, wenn aus einzelnen Datenbanken ein Data Warehouse entsteht? Sascha Dittmann zeigt in dieser Session wie solche und ähnliche Szenarien mit Microsoft Azure umsetzbar sind. Er stellt dabei Technologien wie Azure SQL Database, Azure Elastic Pools und Azure SQL Data Warehouse im Detail vor und gibt Beispiele, sowie Tipps & Tricks aus der Praxis.
Spark auf Hadoop ist hochskalierbar. Cloud Computing ist hochskalierbar. R, die erweiterbare Open Source Data Science Software, eher nicht. Aber was passiert, wenn wir Spark auf Hadoop, Cloud Computing und den Microsoft R Server zu einer skalierbaren Data Science-Plattform zusammenfügen? Stellen Sie sich vor wie es sein könnte, wenn Sie das Erkunden, Transformieren und Modellieren von Daten in jeder beliebigen Größe aus Ihrer Lieblings-R-Umgebung durchführen könnten. Stellen Sie sich nun vor, wie man anschließend die erzeugten Modelle - mit wenigen Klicks - als skalierbare, cloud basierte Web-Services-API bereitstellt. In dieser Session zeigt Sascha Dittmann, wie Sie Ihren R-Code, tausende von Open-Source-R-Pakete sowie die verteilte Implementierungen der beliebtesten Maschine-Learning-Algorithmen nutzen können, um genau dies umzusetzen. Dabei zeigt er wie man ein HDInsight Spark-Cluster inkl. eines Microsoft R Server-Clusters erstellt, sowie das daraus entstandene Model im SQL Server oder als swagger-based API für Anwendungsentwickler bereitstellt.
SQL Server vs. Azure DocumentDB – Ein Battle zwischen XML und JSONSascha Dittmann
Seit dem SQL Server 2000 hielt Stück für Stück die XML-Unterstützung Einzug in die Microsoft RDBMS Welt.
Mit der Azure DocumentDB kam die zweite, hauseigene NoSQL-Datenbank in der Microsoft Cloud hinzu, welche die Daten im JSON-Format verarbeitet.
In dieser Session werden wir anhand eines Praxisbeispiels step-by-step, d.h. von den vorbereitenden Schritten, über das Schreiben bis hin zum Lesen der Daten, diese beiden Technologien gegenüberstellen.
Dabei arbeiten wir die Vor- und Nachteile der einzelnen Ansätze heraus und zeigen Best Practices auf.
Wer hoch-skalierbare Anwendungen in Microsoft Azure bauen wollte, kam bislang an den Azure Cloud Services nicht vorbei. Doch diese Technologie hatte ihre Ecken und Kanten. Die an der BUILD 2015 veröffentlichte Azure Service Fabric Preview soll hierbei Abhilfe schaffen. Mit ihr sollen Entwickler schneller und unkomplizierter Anwendungen für die Cloud, wie auch fürs eigene Rechenzentrum, konstruieren können. Sascha Dittmann durfte bereits seit Ende letzen Jahres mit dem Dienst Erfahrungen sammeln und stellt diesen, anhand einiger Code-Beispiele, in dieser Session vor.
SQL Saturday #313 Rheinland - MapReduce in der PraxisSascha Dittmann
Das Programmiermodel MapReduce, welches vor einigen Jahren von Google veröffentlicht wurde, hat Einzug in zahlreiche Systeme erhalten. Dabei wurde es sowohl als eigenständiges System, wie beispielweise bei Hadoop, Disco oder Amazon Elastic MapReduce, aber auch als Abfragesprache innerhalb größerer Systeme, wie beispielweise bei MongoDB, Greenplum DB oder Aster Data, implementiert. Diese Session stellt gängige Problemstellungen aus der Praxis vor und wie diese mit dem MapReduce Framework von Microsoft HDInsight umgesetzt werden können.
In 2010 stellten die Entwickler von Hadoop fest, dass bei sehr große Clustern (4.000 Knoten und mehr) das bisherige MapReduce Framework nicht mehr richtig skaliert.
Deshalb wurde dieses komplett überarbeitet.
Das Ergebnis war YARN (Yet Another Resource Negotiator).
Neben einer besseren Skalierbarkeit erzeugte YARN weitere positive Nebeneffekte.
Im Oktober 2013 wurde YARN mit dem Hadoop 2.0 Release veröffentlicht.
Was es mit YARN auf sich hat - und welche zusätzlichen Änderungen in Hadoop 2.0 eingeflossen sind - zeigt diese Session.
SQLSaturday #230 - Introduction to Microsoft Big Data (Part 2)Sascha Dittmann
Im zweiten Teil unserer Microsoft Big Data Session geht es darum, wie Big Data Informationen über "klassisches" SQL zugänglich gemacht werden können und wie sich mit der neuen PolyBase-Engine unstrukturierte Hadoop-Daten mit relationalen Data Warehouse-Daten einfach verknüpfen lassen.
In der Hadoop-Welt wird der SQL-Zugriff über die Komponente Hive ermöglicht.
Über den Microsoft Hive ODBC-Konnektor können die üblichen BI-Tools, wie PowerPivot, diesen Zugriff direkt nutzen.
Die PolyBase-Engine schließlich wird ein Bestandteil des SQL Server 2012 Parallel Data Warehouse werden und erlaubt einem transparenten SQL-Zugriff, egal, wo sich die Daten befinden.
SQLSaturday #230 - Introduction to Microsoft Big Data (Part 1)Sascha Dittmann
In dieser Session stellen wir anhand eines praktischen Szenarios vor, wie konkrete Aufgabenstellungen mit HDInsight in der Praxis gelöst werden können:
- Grundlagen von HDInsight für Windows Server und Windows Azure
- Mit Windows Azure HDInsight arbeiten
- MapReduce-Jobs mit Javascript und .NET Code implementieren
Developer Open Space 2012 - Cloud Computing WorkshopSascha Dittmann
Der Einbezug von Cloud Computing in die tägliche Entwicklungsarbeit gewinnt weiter an Bedeutung. Ob hybride Ansätze oder Greenfield-Engineering, Windows Azure bietet die entsprechenden Building Blocks um eine Anforderung zielführend umzusetzen. In dem Workshop werden wir gemeinsam eine klassische on-premise Anwendung „cloudifizieren“ und Schritt für Schritt Cloud-Ressourcen zielführend einsetzen.
.NET Usergroup Rhein-Neckar: Big Data in der Cloud - Apache Hadoop-based Serv...Sascha Dittmann
Wir leben in einem Datenzeitalter! Nach Schätzungen in 2006, betrug das weltweite "Datenuniversum" ca. 0.18 Zettabytes (1 ZB => 10 hoch 21 Bytes bzw. 1 Mrd. Terabytes). In 2011 hatte sich dieses Volumen sogar verzehnfacht (1,8 Zettabytes). Somit wird in vielen Anwendungsszenarien das Thema Big Data und Big Processing immer wichtiger.
Klassische relationale Datenbanksysteme, sowie Statistik und Visualisierungstools, sind oft nicht in der Lage, derart große Datenmengen zu verarbeiten. Für Big Data kommt daher eine neue Art von Software zum Einsatz, die massiv parallel auf bis zu hunderten oder tausenden von Prozessoren bzw. Servern arbeitet.
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfPaige Cruz
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdfPeter Spielvogel
Building better applications for business users with SAP Fiori.
• What is SAP Fiori and why it matters to you
• How a better user experience drives measurable business benefits
• How to get started with SAP Fiori today
• How SAP Fiori elements accelerates application development
• How SAP Build Code includes SAP Fiori tools and other generative artificial intelligence capabilities
• How SAP Fiori paves the way for using AI in SAP apps
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024Neo4j
Neha Bajwa, Vice President of Product Marketing, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...SOFTTECHHUB
The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing.
One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
2. Large Hadron Collider (CERN Schweiz)
http://public.web.cern.ch/public/en/lhc/Computing-en.html
Der LHC Teilchenbeschleuniger
produziert 15 PB Messdaten pro Jahr*
3. Woher kommt Big Data
70% of U.S.
smartphone owners
regularly shop online
via their devices.
44% of users
(350M people)
access Facebook via
mobile devices.
50% of
millennials use
mobile devices to
research products.
60%of U.S.
mobile data will be
audio and video
streaming by 2014.
Mobility
2/3of the world's
mobile data traffic will
be video by 2016.
33%of BI will
be consumed via
handheld devices
by 2013.
Gaming consoles are
now used an average of
1.5 hrs/wk
to connect to the
Internet.
80%growth of
unstructured data is
predicted over the
next five years.
1.8 zettabytes
of digital data were
in use
worldwide in
2011, up 30%
from 2010.
1 in 4
Facebook users
add their location
to posts
(2B/month).
500M Tweets
are hosted on
Twitter each day.
38% of people
recommend a brand
they “like” or follow
on a social network.
100M
Facebook
“likes” per day.
Brands get
Big
Data
Social
Mobility Cloud
4. Big Data Szenarien
Web app
optimization
Smart meter
monitoring
Equipment
monitoring
Advertising
analysis
Life sciences
research
Fraud
detection
Healthcare
outcomes
Weather
forecasting
Natural resource
exploration
Social network
analysis
Churn
analysis
Traffic flow
optimization
IT infrastructure
optimization
Legal
discovery
13. Map/Reduce am Beispiel von Messdaten
0067011990999991950051507004+68750+023550FM-12+038299999V0203301N00671220001CN9999999N9+00001+99999999999
0043011990999991950051512004+68750+023550FM-12+038299999V0203201N00671220001CN9999999N9+00221+99999999999
0043011990999991950051518004+68750+023550FM-12+038299999V0203201N00261220001CN9999999N9-00111+99999999999
0043012650999991949032412004+62300+010750FM-12+048599999V0202701N00461220001CN0500001N9+01111+99999999999
0043012650999991949032418004+62300+010750FM-12+048599999V0202701N00461220001CN0500001N9+00781+99999999999
Jahr Lufttemperatur
14. Map/Reduce am Beispiel von Messdaten
0067011990999991950051507004+68750+023550FM-12+038299999V0203301N00671220001CN9999999N9+00001+99999999999
0043011990999991950051512004+68750+023550FM-12+038299999V0203201N00671220001CN9999999N9+00221+99999999999
0043011990999991950051518004+68750+023550FM-12+038299999V0203201N00261220001CN9999999N9-00111+99999999999
0043012650999991949032412004+62300+010750FM-12+048599999V0202701N00461220001CN0500001N9+01111+99999999999
0043012650999991949032418004+62300+010750FM-12+048599999V0202701N00461220001CN0500001N9+00781+99999999999
Messqualität
32. RDBMS vs. Hadoop
RDBMS Hadoop
Volumen Gigabyte Petabyte
Verarbeitung Ad-Hoc und batch Batch
Updates Viele Lese- und
Schreibzugriffe
Einmal schreiben,
Viele Lesezugriffe
Schema Statisches Schema Dynamisches Schema
Datenintegrität Hoch Niedrig
Skalierverhalten Nicht-Linear Linear