TAROT 2013 9th International Summer School on Training And Research On Testing, Volterra, Italy, 9-13 July, 2013
These slides summarize Myra Cohen's presentation about "Sampling, Re-use and Incremental Testing in Software Product Lines"
Sungho Yoo has over 10 years of experience developing software for mobile devices and set-top boxes. His experience includes developing CDMA modem drivers and protocol stacks for 1x and EV-DO networks in Korea, call user interfaces for Verizon Wireless phones using Qualcomm chips, and messaging apps for AT&T. More recently, he has focused on porting the Adobe Flash Player to various platforms including smart TVs and set-top boxes to enable video capabilities.
El documento propone un proyecto de capacitación docente en el uso de las TIC y plataformas virtuales como Moodle para profesores del colegio "Marcelino Zora Carvajal" en Tacna, Perú. El proyecto tiene como objetivo general capacitar a los docentes en el uso de herramientas computacionales y la plataforma Moodle a través de varios módulos. Se justifica la necesidad de la capacitación debido a que muchos profesores carecen de destrezas informáticas.
Sí eres una empresa y necesitas formar a tus empleados, descarga este manual y accede a nuestros cursos para empresas, con servicios y precios exclusivos.
Aprendeteca.com
A Guide to the SOA Galaxy: Strategy, Design and Best PracticesDmitri Shiryaev
A guide provides an overview of concepts and technologies related to service-oriented architectures (SOA) including:
- SOA aims to create a flexible IT environment that can quickly adapt to changing business needs through loosely coupled reusable services.
- Key SOA technologies discussed include web services, BPEL for orchestrating service interactions, the Java Business Integration (JBI) framework, and enterprise service buses.
- Sun's vision for a SOA platform includes the Java Enterprise System, Java Composite Application Platform Suite, and OpenESB for open source integration and management capabilities.
Úvod do optimalizace konverzních poměrů (CRO)
Psychologie
- Přesvědčivost (persuasion)
- Neuromarketing
- Modality
Vizuální hierarchie
Elementy na webu
Analytika
Analýza výsledků a tvorba hypotézy
A/B a multivariační testování
Personalizace
El documento presenta la asignatura de Microbiología de un programa de Medicina. Cubre temas como datos generales del curso, competencias, contenidos y programación. La asignatura enseña sobre bacterias, hongos y virus que causan enfermedades, con énfasis en identificación, patogenia y epidemiología. Los contenidos se dividen en tres unidades: bacteriología, micología y virología.
Jadual tugas bulanan Felo Unit Kebajikan Pelajar Kolej Uniti untuk bulan Disember 2013 menunjukkan senarai nama Felo dan jadual tugas mereka mengikut minggu. Felo yang bertugas pada minggu tertentu akan bertanggungjawab 24 jam terhadap sebarang kecemasan, mengantar dan mengambil pelajar sakit, serta meronda kawasan kediaman pelajar. Felo yang tidak bertugas masih perlu membantu jika dipanggil unt
Sungho Yoo has over 10 years of experience developing software for mobile devices and set-top boxes. His experience includes developing CDMA modem drivers and protocol stacks for 1x and EV-DO networks in Korea, call user interfaces for Verizon Wireless phones using Qualcomm chips, and messaging apps for AT&T. More recently, he has focused on porting the Adobe Flash Player to various platforms including smart TVs and set-top boxes to enable video capabilities.
El documento propone un proyecto de capacitación docente en el uso de las TIC y plataformas virtuales como Moodle para profesores del colegio "Marcelino Zora Carvajal" en Tacna, Perú. El proyecto tiene como objetivo general capacitar a los docentes en el uso de herramientas computacionales y la plataforma Moodle a través de varios módulos. Se justifica la necesidad de la capacitación debido a que muchos profesores carecen de destrezas informáticas.
Sí eres una empresa y necesitas formar a tus empleados, descarga este manual y accede a nuestros cursos para empresas, con servicios y precios exclusivos.
Aprendeteca.com
A Guide to the SOA Galaxy: Strategy, Design and Best PracticesDmitri Shiryaev
A guide provides an overview of concepts and technologies related to service-oriented architectures (SOA) including:
- SOA aims to create a flexible IT environment that can quickly adapt to changing business needs through loosely coupled reusable services.
- Key SOA technologies discussed include web services, BPEL for orchestrating service interactions, the Java Business Integration (JBI) framework, and enterprise service buses.
- Sun's vision for a SOA platform includes the Java Enterprise System, Java Composite Application Platform Suite, and OpenESB for open source integration and management capabilities.
Úvod do optimalizace konverzních poměrů (CRO)
Psychologie
- Přesvědčivost (persuasion)
- Neuromarketing
- Modality
Vizuální hierarchie
Elementy na webu
Analytika
Analýza výsledků a tvorba hypotézy
A/B a multivariační testování
Personalizace
El documento presenta la asignatura de Microbiología de un programa de Medicina. Cubre temas como datos generales del curso, competencias, contenidos y programación. La asignatura enseña sobre bacterias, hongos y virus que causan enfermedades, con énfasis en identificación, patogenia y epidemiología. Los contenidos se dividen en tres unidades: bacteriología, micología y virología.
Jadual tugas bulanan Felo Unit Kebajikan Pelajar Kolej Uniti untuk bulan Disember 2013 menunjukkan senarai nama Felo dan jadual tugas mereka mengikut minggu. Felo yang bertugas pada minggu tertentu akan bertanggungjawab 24 jam terhadap sebarang kecemasan, mengantar dan mengambil pelajar sakit, serta meronda kawasan kediaman pelajar. Felo yang tidak bertugas masih perlu membantu jika dipanggil unt
Learn how to take event marketing programs to the next level, leveraging social media to maximize the impact of all event efforts--from planning and promotion, to execution and follow-up. This presentation from Constant Contact and HubSpot explores:
• Where to promote events to increase registration
• How to inject social media marketing into events
• How to brand an event for maximum exposure
• What to include in the event follow-up
Using Internet & Smartphone Tools to Engage & Support Volunteers (April 2013,...PDXTech4Good.org
Thanks to the Internet and handheld computers like smart phones, it's never been easier for nonprofits, schools, libraries, government programs and other mission-based initiatives to keep volunteers feeling supported, informed and valued. But how do you communicate all you need to without overwhelming volunteers? How do you appeal to the many different communications styles among volunteers? How do you keep your messages from being drowned out from the growing volume of online "noise" and landslide of information and Internet memes? And how do you integrate online communications with your many different other priorities? This lively discussion will explore all this and more!
Jayne Cravens is an internationally-recognized trainer, researcher and consultant. She is a pioneer regarding the research and practice of virtual volunteering, and she is a veteran manager of various local and international initiatives. She has been quoted in articles in The New York Times, The Wall Street Journal and the Associated Press, as well as for reports by CNN, Deutsche Well, the BBC, and various local radio and TV stations. Resources from her web site, coyotecommunications.com, are frequently cited in reports and articles by a variety of organizations, online and in-print. She is currently based near Portland, Oregon in the USA.
A recording from PDXTech4Good, a free monthly gathering of nonprofits, techies and activists in Portland, Oregon. More information: PDXTech4Good.org
proyecto final IAVA "PAPÁS "AL INFORME COSTANTE DE SUS HIJOS"mariazelara
Este documento describe un proyecto para mejorar la comunicación entre maestros y padres de familia en un jardín de niños mediante el uso de ambientes virtuales. El proyecto implementará un cronograma de actividades que incluye el uso de un webquest para informar a los padres sobre lecciones, tareas y eventos, y videoconferencias para compartir actualizaciones sobre el progreso de los niños. El objetivo es mejorar el involucramiento de los padres y apoyar el aprendizaje de los estudiantes tanto en la escuela como en casa.
El documento explica qué son los píxeles y la resolución en imágenes digitales. Los píxeles son los puntos de color que componen una imagen, y cuantos más píxeles tiene una imagen, mayor es su resolución o detalle. La resolución se mide normalmente en el número de píxeles de ancho y alto de la imagen. Existen diferentes tipos de resolución como la espacial, espectral, radiométrica y temporal. También hay diferentes tipos de píxeles como los muertos, calientes y atascados.
Instalacion Paso A Paso De Linux Redhat Es Varsion 4Edgar Sandoval
Este documento presenta los pasos para instalar la versión 4.0 de Red Hat Enterprise Linux como una máquina virtual, incluyendo insertar el CD de instalación, completar varios pasos en la interfaz gráfica de instalación, e instalar paquetes y el sistema operativo antes de terminar el proceso de instalación.
This document promotes the services of a close-up magician named Zap who performs magic tricks at restaurants to enhance customers' dining experiences. It claims that Zap's services can generate more customer traffic, build loyalty, and give restaurants a competitive advantage by creating positive word-of-mouth advertising. Several successful restaurants that currently employ Zap are listed as examples. Testimonials from restaurant managers praise how Zap's magic entertainment adds value for customers and aligns with their brands. Contact information is provided to book Zap's services.
El documento expresa preocupación por el deterioro de la democracia en Argentina a pesar del tiempo transcurrido desde su restablecimiento. Señala que pese a mantenerse la formalidad democrática, el funcionamiento real del sistema se aleja de los estándares de una auténtica democracia. También advierte sobre los riesgos de fraude electoral por parte del oficialismo debido a su imperiosa necesidad de ganar la próxima elección presidencial para asegurar la impunidad de los Kirchner y sus allegados.
El documento describe una prueba en el mar de la lancha Cobalt 232. La Cobalt 232 acelera rápidamente, alcanzando los 41,1 nudos de velocidad máxima con suavidad. Tiene un comportamiento ágil y estable en giros y contra giros. Su interior ofrece amplio espacio para pasajeros con asientos en la proa y popa.
Fernando Royuela es un escritor español nacido en 1963 en Madrid. Se licenció en derecho y trabaja como abogado, además de escribir novelas. Algunas de sus obras más destacadas son "La mala muerte", por la que ganó el Premio Ojo Crítico, y "Cuando Lázaro anduvo". Royuela escribe para explorar y comprender el mundo de maneras que de otra forma serían inexplicables, así como para expresarse, huir de la realidad y porque disfruta de la escritura.
Raúl ha jugado quince años en el Real Madrid. Marcó 323 goles y debutó a los diecisiete años en un barrio pobre de Madrid llamado Villaverde. Ganó su primera Copa de Europa en 2002 y jugó 102 partidos con la selección española.
The document discusses trends in botnet and DDoS attacks based on a survey of service providers and analysis from Arbor Networks. Some key points:
- The size and frequency of DDoS attacks is increasing, with the largest attack observed reaching 24 Gbps in 2007. On average, there were over 1,000 attacks per day from 2006-2007.
- TCP attacks are most common, followed by UDP floods and application attacks. DNS amplification attacks have increased but remain less prevalent.
- Botnets are the top concern for service providers. They are used for DDoS attacks, spam, and hosting malware. New botnets are becoming more targeted, encrypted, and distributed.
- There
Workshop: Recherche 2.0 - So funktioniert der Faktencheck im (Social) WebAndré Vatter
Um langfristig im journalistischen und kommunikativen Wettbewerb zu bestehen, müssen Journalisten, PR-Fachleute und Kommunikationsschaffende in der Lage sein, Informationen im Web schnell ermitteln, bewerten und validieren zu können. Das Seminar vermittelt hierfür das notwendige Know-how. Es wird erläutert, warum Google und Wikipedia nicht das Maß aller Dinge sind, welche Tools eine professionelle und effiziente Recherche sowie ein Monitoring im Web 2.0 vereinfachen.
Polonium is a rare, radioactive metalloid discovered in 1898 by Marie and Pierre Curie. It has no stable isotopes and emits blue light as its radioactively decaying isotopes excite the surrounding air. Polonium is produced through neutron or proton bombardment of bismuth and has uses in nuclear batteries, heat sources, and eliminating static electricity, though it is also highly toxic due to its radioactivity and chemical properties.
It can be a daunting challenge for an organization to transition from an embedded email culture to a more structured solution like JIRA Service Desk. This problem is compounded when external partners are involved. Through practical customer examples, we will cover how to iteratively graduate from email support, and set up a full-blown service desk.
This talk was presented by Tom Moors at the Atlassian Summit 2014 on Sept 10, 2:00 PM – 2:20 PM.
La Edad Moderna se define como el periodo entre los siglos XV y XVIII, caracterizado por el triunfo de los valores de la modernidad como el progreso, la razón y la comunicación. Fue una época de grandes descubrimientos geográficos, revoluciones científicas, y conflictos religiosos como la Reforma Protestante. También se produjeron cambios sociales y económicos importantes, como el surgimiento del capitalismo y la burguesía.
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2022/06/seamless-deployment-of-multimedia-and-machine-learning-applications-at-the-edge-a-presentation-from-qualcomm/
Megha Daga, Senior Director of Product Management for AIoT at Qualcomm, presents the “Seamless Deployment of Multimedia and Machine Learning Applications at the Edge” tutorial at the May 2022 Embedded Vision Summit.
There has been an explosion of opportunities for edge compute solutions across the internet of things. This growth in opportunities and the diversity of applications is leading to fragmentation in the IoT space both in hardware and software, which creates challenges for developers. In addition, customers and developers are facing challenges in efficient data management and optimized application deployment on embedded edge platforms.
In this session, Daga introduces the Qualcomm Intelligent Multimedia SDK, which empowers developers to tackle these challenges and deploy edge compute applications in a scalable, flexible and optimized way. The Qualcomm Intelligent Multimedia SDK easily decodes and organizes sensor data and executes applications efficiently on edge platforms.
FutureComm 2010: Video Quality Analysis and MeasurementRADVISION Ltd.
This document summarizes RADVISION's video quality solutions. It discusses (1) factors that affect video quality like network conditions and content complexity, (2) RADVISION's video quality SDK and standalone testing suite that allow measuring and analyzing video quality in real-time, and (3) RADVISION's innovative research in areas like human vision databases and synchronization algorithms. The document emphasizes RADVISION's leadership in video quality assessment based on 17 years of experience in video conferencing.
Maintaining the Front Door to Netflix : The Netflix APIDaniel Jacobson
This presentation was given to the engineering organization at Zendesk. In this presentation, I talk about the challenges that the Netflix API faces in supporting the 1000+ different device types, millions of users, and billions of transactions. The topics range from resiliency, scale, API design, failure injection, continuous delivery, and more.
Developing Apps for Windows Phone 8 - The 5th seminar in public seminar series from KMS Technology which have been delivering from 2011 in every two months
This document summarizes AT&T Research's participation in the 2009 TREC Video Retrieval Evaluation content-based copy detection task. It describes their approaches to shot boundary detection, transformation detection and normalization of query keyframes, indexing reference videos using locality sensitive hashing, and refining matches through RANSAC and scoring techniques. Their system was evaluated on a dataset containing over 1,000 query videos and 800 reference videos, extracting over 5 million features. Results showed their technique could accurately detect copied video segments in the dataset.
Learn how to take event marketing programs to the next level, leveraging social media to maximize the impact of all event efforts--from planning and promotion, to execution and follow-up. This presentation from Constant Contact and HubSpot explores:
• Where to promote events to increase registration
• How to inject social media marketing into events
• How to brand an event for maximum exposure
• What to include in the event follow-up
Using Internet & Smartphone Tools to Engage & Support Volunteers (April 2013,...PDXTech4Good.org
Thanks to the Internet and handheld computers like smart phones, it's never been easier for nonprofits, schools, libraries, government programs and other mission-based initiatives to keep volunteers feeling supported, informed and valued. But how do you communicate all you need to without overwhelming volunteers? How do you appeal to the many different communications styles among volunteers? How do you keep your messages from being drowned out from the growing volume of online "noise" and landslide of information and Internet memes? And how do you integrate online communications with your many different other priorities? This lively discussion will explore all this and more!
Jayne Cravens is an internationally-recognized trainer, researcher and consultant. She is a pioneer regarding the research and practice of virtual volunteering, and she is a veteran manager of various local and international initiatives. She has been quoted in articles in The New York Times, The Wall Street Journal and the Associated Press, as well as for reports by CNN, Deutsche Well, the BBC, and various local radio and TV stations. Resources from her web site, coyotecommunications.com, are frequently cited in reports and articles by a variety of organizations, online and in-print. She is currently based near Portland, Oregon in the USA.
A recording from PDXTech4Good, a free monthly gathering of nonprofits, techies and activists in Portland, Oregon. More information: PDXTech4Good.org
proyecto final IAVA "PAPÁS "AL INFORME COSTANTE DE SUS HIJOS"mariazelara
Este documento describe un proyecto para mejorar la comunicación entre maestros y padres de familia en un jardín de niños mediante el uso de ambientes virtuales. El proyecto implementará un cronograma de actividades que incluye el uso de un webquest para informar a los padres sobre lecciones, tareas y eventos, y videoconferencias para compartir actualizaciones sobre el progreso de los niños. El objetivo es mejorar el involucramiento de los padres y apoyar el aprendizaje de los estudiantes tanto en la escuela como en casa.
El documento explica qué son los píxeles y la resolución en imágenes digitales. Los píxeles son los puntos de color que componen una imagen, y cuantos más píxeles tiene una imagen, mayor es su resolución o detalle. La resolución se mide normalmente en el número de píxeles de ancho y alto de la imagen. Existen diferentes tipos de resolución como la espacial, espectral, radiométrica y temporal. También hay diferentes tipos de píxeles como los muertos, calientes y atascados.
Instalacion Paso A Paso De Linux Redhat Es Varsion 4Edgar Sandoval
Este documento presenta los pasos para instalar la versión 4.0 de Red Hat Enterprise Linux como una máquina virtual, incluyendo insertar el CD de instalación, completar varios pasos en la interfaz gráfica de instalación, e instalar paquetes y el sistema operativo antes de terminar el proceso de instalación.
This document promotes the services of a close-up magician named Zap who performs magic tricks at restaurants to enhance customers' dining experiences. It claims that Zap's services can generate more customer traffic, build loyalty, and give restaurants a competitive advantage by creating positive word-of-mouth advertising. Several successful restaurants that currently employ Zap are listed as examples. Testimonials from restaurant managers praise how Zap's magic entertainment adds value for customers and aligns with their brands. Contact information is provided to book Zap's services.
El documento expresa preocupación por el deterioro de la democracia en Argentina a pesar del tiempo transcurrido desde su restablecimiento. Señala que pese a mantenerse la formalidad democrática, el funcionamiento real del sistema se aleja de los estándares de una auténtica democracia. También advierte sobre los riesgos de fraude electoral por parte del oficialismo debido a su imperiosa necesidad de ganar la próxima elección presidencial para asegurar la impunidad de los Kirchner y sus allegados.
El documento describe una prueba en el mar de la lancha Cobalt 232. La Cobalt 232 acelera rápidamente, alcanzando los 41,1 nudos de velocidad máxima con suavidad. Tiene un comportamiento ágil y estable en giros y contra giros. Su interior ofrece amplio espacio para pasajeros con asientos en la proa y popa.
Fernando Royuela es un escritor español nacido en 1963 en Madrid. Se licenció en derecho y trabaja como abogado, además de escribir novelas. Algunas de sus obras más destacadas son "La mala muerte", por la que ganó el Premio Ojo Crítico, y "Cuando Lázaro anduvo". Royuela escribe para explorar y comprender el mundo de maneras que de otra forma serían inexplicables, así como para expresarse, huir de la realidad y porque disfruta de la escritura.
Raúl ha jugado quince años en el Real Madrid. Marcó 323 goles y debutó a los diecisiete años en un barrio pobre de Madrid llamado Villaverde. Ganó su primera Copa de Europa en 2002 y jugó 102 partidos con la selección española.
The document discusses trends in botnet and DDoS attacks based on a survey of service providers and analysis from Arbor Networks. Some key points:
- The size and frequency of DDoS attacks is increasing, with the largest attack observed reaching 24 Gbps in 2007. On average, there were over 1,000 attacks per day from 2006-2007.
- TCP attacks are most common, followed by UDP floods and application attacks. DNS amplification attacks have increased but remain less prevalent.
- Botnets are the top concern for service providers. They are used for DDoS attacks, spam, and hosting malware. New botnets are becoming more targeted, encrypted, and distributed.
- There
Workshop: Recherche 2.0 - So funktioniert der Faktencheck im (Social) WebAndré Vatter
Um langfristig im journalistischen und kommunikativen Wettbewerb zu bestehen, müssen Journalisten, PR-Fachleute und Kommunikationsschaffende in der Lage sein, Informationen im Web schnell ermitteln, bewerten und validieren zu können. Das Seminar vermittelt hierfür das notwendige Know-how. Es wird erläutert, warum Google und Wikipedia nicht das Maß aller Dinge sind, welche Tools eine professionelle und effiziente Recherche sowie ein Monitoring im Web 2.0 vereinfachen.
Polonium is a rare, radioactive metalloid discovered in 1898 by Marie and Pierre Curie. It has no stable isotopes and emits blue light as its radioactively decaying isotopes excite the surrounding air. Polonium is produced through neutron or proton bombardment of bismuth and has uses in nuclear batteries, heat sources, and eliminating static electricity, though it is also highly toxic due to its radioactivity and chemical properties.
It can be a daunting challenge for an organization to transition from an embedded email culture to a more structured solution like JIRA Service Desk. This problem is compounded when external partners are involved. Through practical customer examples, we will cover how to iteratively graduate from email support, and set up a full-blown service desk.
This talk was presented by Tom Moors at the Atlassian Summit 2014 on Sept 10, 2:00 PM – 2:20 PM.
La Edad Moderna se define como el periodo entre los siglos XV y XVIII, caracterizado por el triunfo de los valores de la modernidad como el progreso, la razón y la comunicación. Fue una época de grandes descubrimientos geográficos, revoluciones científicas, y conflictos religiosos como la Reforma Protestante. También se produjeron cambios sociales y económicos importantes, como el surgimiento del capitalismo y la burguesía.
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2022/06/seamless-deployment-of-multimedia-and-machine-learning-applications-at-the-edge-a-presentation-from-qualcomm/
Megha Daga, Senior Director of Product Management for AIoT at Qualcomm, presents the “Seamless Deployment of Multimedia and Machine Learning Applications at the Edge” tutorial at the May 2022 Embedded Vision Summit.
There has been an explosion of opportunities for edge compute solutions across the internet of things. This growth in opportunities and the diversity of applications is leading to fragmentation in the IoT space both in hardware and software, which creates challenges for developers. In addition, customers and developers are facing challenges in efficient data management and optimized application deployment on embedded edge platforms.
In this session, Daga introduces the Qualcomm Intelligent Multimedia SDK, which empowers developers to tackle these challenges and deploy edge compute applications in a scalable, flexible and optimized way. The Qualcomm Intelligent Multimedia SDK easily decodes and organizes sensor data and executes applications efficiently on edge platforms.
FutureComm 2010: Video Quality Analysis and MeasurementRADVISION Ltd.
This document summarizes RADVISION's video quality solutions. It discusses (1) factors that affect video quality like network conditions and content complexity, (2) RADVISION's video quality SDK and standalone testing suite that allow measuring and analyzing video quality in real-time, and (3) RADVISION's innovative research in areas like human vision databases and synchronization algorithms. The document emphasizes RADVISION's leadership in video quality assessment based on 17 years of experience in video conferencing.
Maintaining the Front Door to Netflix : The Netflix APIDaniel Jacobson
This presentation was given to the engineering organization at Zendesk. In this presentation, I talk about the challenges that the Netflix API faces in supporting the 1000+ different device types, millions of users, and billions of transactions. The topics range from resiliency, scale, API design, failure injection, continuous delivery, and more.
Developing Apps for Windows Phone 8 - The 5th seminar in public seminar series from KMS Technology which have been delivering from 2011 in every two months
This document summarizes AT&T Research's participation in the 2009 TREC Video Retrieval Evaluation content-based copy detection task. It describes their approaches to shot boundary detection, transformation detection and normalization of query keyframes, indexing reference videos using locality sensitive hashing, and refining matches through RANSAC and scoring techniques. Their system was evaluated on a dataset containing over 1,000 query videos and 800 reference videos, extracting over 5 million features. Results showed their technique could accurately detect copied video segments in the dataset.
MIPI DevCon 2016: Image Sensor and Display Connectivity DisruptionMIPI Alliance
The ability to leverage mobile technologies into new consumer, medical, industrial, and automotive markets creates challenges in image sensor and display interfacing. When interface types or number of the interfaces do not match between image sensors, displays and processors, a bridge is required to enable such connectivity. In this presentation, Grant Jennings of Lattice Semiconductor describes connectivity through programmable interface bridges to aid in the development of these systems that were unforeseen or previously could not be rationalized.
This document outlines the product and marketing plans for 2011 for a CCTV equipment company. It discusses the team members and their responsibilities in areas such as generating sales opportunities, inventory management, product support, and vendor relationship management. It also analyzes the company's main products and competitors, identifies target customer groups, and provides projections for sales and values. The overall aim is to seek new customers and sales channels while managing products and vendors to help grow the business in 2011.
Josh Evans, a former engineering leader at Netflix, gave a talk on mastering chaos with microservices at Netflix. He began with introductions and an overview of Netflix's architecture evolution from monoliths to microservices. He then discussed challenges of microservices like dependencies, scale, variance and change. For each challenge, he provided examples of how Netflix addresses issues like cascading failures, operational drift, polyglot environments and intentional variance. Finally, he emphasized that organization must follow architecture through principles like Conway's Law, and that outcomes include both technical solutions and realigning teams.
Mastering Chaos - A Netflix Guide to MicroservicesJosh Evans
QConSF 2016 Abstract:
By embracing the tension between order and chaos and applying a healthy mix of discipline and surrender Netflix reliably operates microservices in the cloud at scale. But every lesson learned and solution developed over the last seven years was born out of pain for us and our customers. Even today we remain vigilant as we evolve our service architecture. For those just starting the microservices journey these lessons and solutions provide a blueprint for success.
In this talk we’ll explore the chaotic and vibrant world of microservices at Netflix. We’ll start with the basics - the anatomy of a microservice, the challenges around distributed systems, and the benefits realized when integrated operational practices and technical solutions are properly leveraged. Then we’ll build on that foundation exploring the cultural, architectural, and operational methods that lead to microservice mastery.
Bug fixing is the most outdated area of web application development. End users of web applications face bugs that are extremely hard for developers to investigate. Kuoll-script confidentially records all information developers need for bug fixing. Having the Kuoll-record a developer can fix the bug without even reproducing it. It made possible with Kuoll-record containing the code of the web page, all dynamic changes of that code, user actions, network requests, and (!) cause-effect relations of all of the above.
Developers spend their time inefficiently, customers are not happy and the company is losing money. Stop it! Install Kuoll now and make bug fixing of end user bugs as easy as in developer's own web browser. http://www.kuoll.com/
Clipper: A Low-Latency Online Prediction Serving SystemDatabricks
Machine learning is being deployed in a growing number of applications which demand real-time, accurate, and robust predictions under heavy serving loads. However, most machine learning frameworks and systems only address model training and not deployment.
Clipper is a general-purpose model-serving system that addresses these challenges. Interposing between applications that consume predictions and the machine-learning models that produce predictions, Clipper simplifies the model deployment process by isolating models in their own containers and communicating with them over a lightweight RPC system. This architecture allows models to be deployed for serving in the same runtime environment as that used during training. Further, it provides simple mechanisms for scaling out models to meet increased throughput demands and performing fine-grained physical resource allocation for each model.
In this talk, I will provide an overview of the Clipper serving system and then discuss how to get started using Clipper to serve Spark and TensorFlow models in a production serving environment.
Voice and Video over IP Communications: Assessing and Improving User ExperienceRADVISION Ltd.
While video deployment over IP is experiencing a significant boom, overall user experience does not always live up to expectations. RADVISION experts will discuss unique, no-reference, video measurement and analysis algorithms that improve user experience for voice and video over IP communications.
By attending this webinar, you will learn:
* Factors that can affects Video Quality
* Different methodology to measure Video Quality
* RADVISION solutions for Video Quality
Who should attend: professionals in visual communication field
* IT managers
* System architects
* Product Managers
* CTOs
* VP R&D
* Telecom operators representatives
Special thanks to Daniel Egan. This presentation provides an introduction to Windows Phone 7, helps developers decide which user controls (Pivot, Panorama), user interface concerns, and step to get your app in Marketplace.
- The document proposes adding stateful capabilities to OpenFlow 2.0 in order to better support use cases that require stateful processing like APS and load balancing.
- It suggests including a stateful flow table, programmable state machines, and the ability to generate and process frames within OpenFlow switches.
- To ensure interoperability, it recommends adopting a bytecode approach where any programming language could be used to define state machines which are then compiled to a common bytecode format and run on switches from different vendors.
Incremental Queries and Transformations for Engineering Critical SystemsÁkos Horváth
This document discusses incremental queries and transformations for engineering critical systems. It describes how model transformations can be used in critical systems engineering to enable early validation of system models. It presents EMF-IncQuery and VIATRA, which allow for incremental queries and transformations over models. These technologies have been applied in various industrial domains including avionics, automotive, and telecommunications. The talk concludes by discussing some of the industrial applications and contributors to this work.
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/whats-next-in-on-device-generative-ai-a-presentation-from-qualcomm/
Jilei Hou, Vice President of Engineering and Head of AI Research at Qualcomm Technologies, presents the “What’s Next in On-device Generative AI” tutorial at the May 2024 Embedded Vision Summit.
The generative AI era has begun! Large multimodal models are bringing the power of language understanding to machine perception, and transformer models are expanding to allow machines to understand using multiple types of sensors. This new wave of approaches is poised to revolutionize user experiences, disrupt industries and enable powerful new capabilities. For generative AI to reach its full potential, however, we must deploy it on edge devices, providing improved latency, pervasive interaction and enhanced privacy.
In this talk, Hou shares Qualcomm’s vision of the compelling opportunities enabled by efficient generative AI at the edge. He also identifies the key challenges that the industry must overcome to realize the massive potential of these technologies. And he highlights research and product development work that Qualcomm is doing to lead the way via an end-to-end system approach—including techniques for efficient on-device execution of LLMs, LVMs and LMMs, methods for orchestration of large models at the edge and approaches for adaptation and personalization.
SignalR powered real-time x-plat mobile apps!Sam Basu
This document discusses enabling real-time functionality in mobile apps using SignalR.
It begins by introducing SignalR and how it provides an asynchronous and persistent connection between server and clients for bidirectional communication. It then discusses challenges with existing polling techniques and how SignalR abstracts away the network transport layer.
The document also covers SignalR platform support on both server and client sides, its APIs, and examples of how it has been used to power chat applications and games. It explores using SignalR with mobile clients via technologies like Telerik AppBuilder and Xamarin to build cross-platform hybrid mobile apps with real-time capabilities.
The document summarizes a PhD thesis defense about novel applications for emerging markets using television as a ubiquitous device. The thesis proposed improving the user experience of TV as a low-cost internet access device through:
1) QoS-aware video transmissions and low-complexity video security for applications like video chat and distance education.
2) Context-aware intelligent TV-internet mashups using TV channel identity and text recognition from static and broadcast video as context.
3) A novel on-screen keyboard for text entry using a TV remote for non-computer savvy users. User studies on early prototypes identified challenges with interfaces, video quality, and text entry that the thesis aimed to address.
Similar to TAROT2013 Testing School - Myra Cohen presentation (20)
These slides have been presented at the ICSE 2020 conference, SEIS (software engineering in society) track. It reports on our experience within the Uffizi Project, and how we had to take into account human behaiour to design our IoT-based solution.
How cultural heritage, cyber-physical spaces, and software engineering can wo...Henry Muccini
This is a seminar provided to a PhD school on Cultural Heritage Conservation and Valorization.
The focus has been on the interdisciplinarity among cultural heritage, cyber-physical spaces, and software engineering.
Turismo 4.0: l'ICT a supporto del turismo sostenibileHenry Muccini
The importance of sustainable tourism is today very clear, as also highlighted by some national and international organizations. This presentation highlights the role of ICT in the context of sustainable tourism. Some ongoing projects are presented as well.
Sustainable Tourism - IoT and crowd managementHenry Muccini
What is Sustainable Tourism and how IoT may help to reduce crowd management. This material reports on our experience within the Uffizi Galleries project and the CAPS IoT modeling and simulation framework.
Software Engineering at the age of the Internet of ThingsHenry Muccini
This is an overview on Sw Engineering the IoT, created for the FOI, Faculty of Organization and Informatics of the University of Zagreb, and presented during their International Days.
The influence of Group Decision Making on Architecture Design DecisionsHenry Muccini
Group Decision Making influcencs Architecture Design Decisions. This presentation, given as a keynote at the MARCH 2019 workshop (https://is.ieis.tue.nl/research/bpm/MARCH/index.php/keynote/), tries to identifies GDM factors that influence architecture design decisions.
Web Engineering L8: User-centered Design (8/8)Henry Muccini
This lecture focusses on User-centered Design (UCD). It covers the "The Elements of User Experience" book by Garrett.
The topics covered are:
- the UCD process
- Personas
- Scope
- Information Architecture
- Sitemaps
- Wireframes
- Prototypes
The output of this course consists in a list of artifacts and principles to be used when engineering Web applications. The list is availabe at https://trello.com/b/z49P8z3b
Web Engineering L7: Sequence Diagrams and Design Decisions (7/8)Henry Muccini
This lecture covers Sequence diagrams and Design decision models. It covers:
- sequence diagrams in UML 2.x
- the QOC model for design decisions
The output of this course consists in a list of artifacts and principles to be used when engineering Web applications. They are listed at https://trello.com/b/z49P8z3b
Web Engineering L6: Software Architecture for the Web (6/8)Henry Muccini
This lecture discusses Architectural aspects of Web engineering.
It covers:
- software architecture design
- software architecture for the web
- component model for software architecture description
The output of this course consists in a list of artifacts and principles to be used when engineering Web applications is listed at https://trello.com/b/z49P8z3b
Web Engineering L5: Content Model (5/8)Henry Muccini
This lecture focusses on Content Design.
It presents the UWE approach for producing the:
- Conceptual Model
- Navigation Space Model
- Navigational Structure Model
The output of this course consists in a list of artifacts and principles to be used when engineering Web applications is listed at https://trello.com/b/z49P8z3b
Web Engineering L3: Project Planning (3/8)Henry Muccini
This document discusses planning work for a team developing a web application. It covers creating a project plan, work breakdown structure, Gantt charts, critical paths, tracking progress, and risk management. The project plan sets resources, work items, and a schedule. Work is broken down into activities with dependencies. Gantt charts and PERT charts can represent activities graphically. Tracking ensures the project stays on schedule and budget. Risk management identifies potential issues and ways to mitigate them. Project management tools like MS Project, kanban boards, and free online options can aid planning and tracking work for a team.
Web Engineering L2: Requirements Elicitation for the Web (2/8)Henry Muccini
This lecture focusses on requirements elicitation.
It covers:
- Requirements discovery
- Requirements classification
- Requirements Prioritization
- Requirements Specifications
The output of this course consists in a list of artifacts and principles to be used when engineering Web applications is listed at https://trello.com/b/z49P8z3b
Web Engineering L1: introduction to Web Engineering (1/8)Henry Muccini
This lecture makes an introduction to Web Engineering.
- Why web engineering
- Quality
- Issues to avoid
- Web architectures
The output of this course consists in a list of artifacts and principles to be used when engineering Web applications is listed at https://trello.com/b/z49P8z3b
Web Engineering L4: Requirements and Planning in concrete (4/8)Henry Muccini
This lecture summarizes and extends L3, with a focus on:
- Critical Path
- Agile for Planning
- Convergence and divergence
The output of this course consists in a list of artifacts and principles to be used when engineering Web applications is listed at https://trello.com/b/z49P8z3b
Collaborative aspects of Decision Making and its impact on SustainabilityHenry Muccini
In this talk I made an effort to link together sustainability, architecture design decision, and group decision making. Take a look and contact me for questions.
This presentation proposes CAPS, an architecture-driven
modeling framework for the development of Situational Aware
Cyber-Physical Systems.
Situational Awareness involves being aware of what is
happening in the surroundings, and using this information
to decide and act. It has been recognized as a critical,
yet often elusive, foundation for successful decision-making
in complex systems. With the advent of cyber-physical systems
(CPS), situational awareness is playing an increasingly
important role especially in crowd and fleets management,
infrastructure monitoring, and smart city applications. While
specializing cyber physical systems, Situational Aware CPS
requires the continuous monitoring of environmental conditions
and events with respect to time and space. New architectural
concerns arise, especially related to the sense , compute &
communication paradigm, the use of domain-specific hardware
components, and the cyber-physical space dimension.
This work illustrates the CAPS modeling languages used
to describe the software architecture, hardware configuration,
and physical space views for a situational aware CPS.
I progetti UnivAq-UFFIZI, INCIPICT, e CUSPISHenry Muccini
Alcuni progetti dell'Universita' degli Studi dell'Aquila volti al supporto dei beni culturali. Tale presentazione e' stata fornita nel contesto dell'evento Le Gallerie degli Uffizi incontrano UnivAq
Exploring the Temporal Aspects of Software ArchitectureHenry Muccini
The keynote lecture video is now available at http://www.icsoft.org/KeynoteSpeakers.aspx?y=2016
This presentation covers the main topics discussed by the software architecture conferences in the past 15+ years. It provides a systematic, unbiased view on research trends with reflections on the future challenges.
This speech has been provided as a keynote at ICSOFT 2016.
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...DanBrown980551
This LF Energy webinar took place June 20, 2024. It featured:
-Alex Thornton, LF Energy
-Hallie Cramer, Google
-Daniel Roesler, UtilityAPI
-Henry Richardson, WattTime
In response to the urgency and scale required to effectively address climate change, open source solutions offer significant potential for driving innovation and progress. Currently, there is a growing demand for standardization and interoperability in energy data and modeling. Open source standards and specifications within the energy sector can also alleviate challenges associated with data fragmentation, transparency, and accessibility. At the same time, it is crucial to consider privacy and security concerns throughout the development of open source platforms.
This webinar will delve into the motivations behind establishing LF Energy’s Carbon Data Specification Consortium. It will provide an overview of the draft specifications and the ongoing progress made by the respective working groups.
Three primary specifications will be discussed:
-Discovery and client registration, emphasizing transparent processes and secure and private access
-Customer data, centering around customer tariffs, bills, energy usage, and full consumption disclosure
-Power systems data, focusing on grid data, inclusive of transmission and distribution networks, generation, intergrid power flows, and market settlement data
Discover top-tier mobile app development services, offering innovative solutions for iOS and Android. Enhance your business with custom, user-friendly mobile applications.
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsDianaGray10
Join us to learn how UiPath Apps can directly and easily interact with prebuilt connectors via Integration Service--including Salesforce, ServiceNow, Open GenAI, and more.
The best part is you can achieve this without building a custom workflow! Say goodbye to the hassle of using separate automations to call APIs. By seamlessly integrating within App Studio, you can now easily streamline your workflow, while gaining direct access to our Connector Catalog of popular applications.
We’ll discuss and demo the benefits of UiPath Apps and connectors including:
Creating a compelling user experience for any software, without the limitations of APIs.
Accelerating the app creation process, saving time and effort
Enjoying high-performance CRUD (create, read, update, delete) operations, for
seamless data management.
Speakers:
Russell Alfeche, Technology Leader, RPA at qBotic and UiPath MVP
Charlie Greenberg, host
Main news related to the CCS TSI 2023 (2023/1695)Jakub Marek
An English 🇬🇧 translation of a presentation to the speech I gave about the main changes brought by CCS TSI 2023 at the biggest Czech conference on Communications and signalling systems on Railways, which was held in Clarion Hotel Olomouc from 7th to 9th November 2023 (konferenceszt.cz). Attended by around 500 participants and 200 on-line followers.
The original Czech 🇨🇿 version of the presentation can be found here: https://www.slideshare.net/slideshow/hlavni-novinky-souvisejici-s-ccs-tsi-2023-2023-1695/269688092 .
The videorecording (in Czech) from the presentation is available here: https://youtu.be/WzjJWm4IyPk?si=SImb06tuXGb30BEH .
Taking AI to the Next Level in Manufacturing.pdfssuserfac0301
Read Taking AI to the Next Level in Manufacturing to gain insights on AI adoption in the manufacturing industry, such as:
1. How quickly AI is being implemented in manufacturing.
2. Which barriers stand in the way of AI adoption.
3. How data quality and governance form the backbone of AI.
4. Organizational processes and structures that may inhibit effective AI adoption.
6. Ideas and approaches to help build your organization's AI strategy.
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...Alex Pruden
Folding is a recent technique for building efficient recursive SNARKs. Several elegant folding protocols have been proposed, such as Nova, Supernova, Hypernova, Protostar, and others. However, all of them rely on an additively homomorphic commitment scheme based on discrete log, and are therefore not post-quantum secure. In this work we present LatticeFold, the first lattice-based folding protocol based on the Module SIS problem. This folding protocol naturally leads to an efficient recursive lattice-based SNARK and an efficient PCD scheme. LatticeFold supports folding low-degree relations, such as R1CS, as well as high-degree relations, such as CCS. The key challenge is to construct a secure folding protocol that works with the Ajtai commitment scheme. The difficulty, is ensuring that extracted witnesses are low norm through many rounds of folding. We present a novel technique using the sumcheck protocol to ensure that extracted witnesses are always low norm no matter how many rounds of folding are used. Our evaluation of the final proof system suggests that it is as performant as Hypernova, while providing post-quantum security.
Paper Link: https://eprint.iacr.org/2024/257
Introduction of Cybersecurity with OSS at Code Europe 2024Hiroshi SHIBATA
I develop the Ruby programming language, RubyGems, and Bundler, which are package managers for Ruby. Today, I will introduce how to enhance the security of your application using open-source software (OSS) examples from Ruby and RubyGems.
The first topic is CVE (Common Vulnerabilities and Exposures). I have published CVEs many times. But what exactly is a CVE? I'll provide a basic understanding of CVEs and explain how to detect and handle vulnerabilities in OSS.
Next, let's discuss package managers. Package managers play a critical role in the OSS ecosystem. I'll explain how to manage library dependencies in your application.
I'll share insights into how the Ruby and RubyGems core team works to keep our ecosystem safe. By the end of this talk, you'll have a better understanding of how to safeguard your code.
"Scaling RAG Applications to serve millions of users", Kevin GoedeckeFwdays
How we managed to grow and scale a RAG application from zero to thousands of users in 7 months. Lessons from technical challenges around managing high load for LLMs, RAGs and Vector databases.
Northern Engraving | Modern Metal Trim, Nameplates and Appliance PanelsNorthern Engraving
What began over 115 years ago as a supplier of precision gauges to the automotive industry has evolved into being an industry leader in the manufacture of product branding, automotive cockpit trim and decorative appliance trim. Value-added services include in-house Design, Engineering, Program Management, Test Lab and Tool Shops.
inQuba Webinar Mastering Customer Journey Management with Dr Graham HillLizaNolte
HERE IS YOUR WEBINAR CONTENT! 'Mastering Customer Journey Management with Dr. Graham Hill'. We hope you find the webinar recording both insightful and enjoyable.
In this webinar, we explored essential aspects of Customer Journey Management and personalization. Here’s a summary of the key insights and topics discussed:
Key Takeaways:
Understanding the Customer Journey: Dr. Hill emphasized the importance of mapping and understanding the complete customer journey to identify touchpoints and opportunities for improvement.
Personalization Strategies: We discussed how to leverage data and insights to create personalized experiences that resonate with customers.
Technology Integration: Insights were shared on how inQuba’s advanced technology can streamline customer interactions and drive operational efficiency.
High performance Serverless Java on AWS- GoTo Amsterdam 2024Vadym Kazulkin
Java is for many years one of the most popular programming languages, but it used to have hard times in the Serverless community. Java is known for its high cold start times and high memory footprint, comparing to other programming languages like Node.js and Python. In this talk I'll look at the general best practices and techniques we can use to decrease memory consumption, cold start times for Java Serverless development on AWS including GraalVM (Native Image) and AWS own offering SnapStart based on Firecracker microVM snapshot and restore and CRaC (Coordinated Restore at Checkpoint) runtime hooks. I'll also provide a lot of benchmarking on Lambda functions trying out various deployment package sizes, Lambda memory settings, Java compilation options and HTTP (a)synchronous clients and measure their impact on cold and warm start times.
Monitoring and Managing Anomaly Detection on OpenShift.pdfTosin Akinosho
Monitoring and Managing Anomaly Detection on OpenShift
Overview
Dive into the world of anomaly detection on edge devices with our comprehensive hands-on tutorial. This SlideShare presentation will guide you through the entire process, from data collection and model training to edge deployment and real-time monitoring. Perfect for those looking to implement robust anomaly detection systems on resource-constrained IoT/edge devices.
Key Topics Covered
1. Introduction to Anomaly Detection
- Understand the fundamentals of anomaly detection and its importance in identifying unusual behavior or failures in systems.
2. Understanding Edge (IoT)
- Learn about edge computing and IoT, and how they enable real-time data processing and decision-making at the source.
3. What is ArgoCD?
- Discover ArgoCD, a declarative, GitOps continuous delivery tool for Kubernetes, and its role in deploying applications on edge devices.
4. Deployment Using ArgoCD for Edge Devices
- Step-by-step guide on deploying anomaly detection models on edge devices using ArgoCD.
5. Introduction to Apache Kafka and S3
- Explore Apache Kafka for real-time data streaming and Amazon S3 for scalable storage solutions.
6. Viewing Kafka Messages in the Data Lake
- Learn how to view and analyze Kafka messages stored in a data lake for better insights.
7. What is Prometheus?
- Get to know Prometheus, an open-source monitoring and alerting toolkit, and its application in monitoring edge devices.
8. Monitoring Application Metrics with Prometheus
- Detailed instructions on setting up Prometheus to monitor the performance and health of your anomaly detection system.
9. What is Camel K?
- Introduction to Camel K, a lightweight integration framework built on Apache Camel, designed for Kubernetes.
10. Configuring Camel K Integrations for Data Pipelines
- Learn how to configure Camel K for seamless data pipeline integrations in your anomaly detection workflow.
11. What is a Jupyter Notebook?
- Overview of Jupyter Notebooks, an open-source web application for creating and sharing documents with live code, equations, visualizations, and narrative text.
12. Jupyter Notebooks with Code Examples
- Hands-on examples and code snippets in Jupyter Notebooks to help you implement and test anomaly detection models.
Northern Engraving | Nameplate Manufacturing Process - 2024Northern Engraving
Manufacturing custom quality metal nameplates and badges involves several standard operations. Processes include sheet prep, lithography, screening, coating, punch press and inspection. All decoration is completed in the flat sheet with adhesive and tooling operations following. The possibilities for creating unique durable nameplates are endless. How will you create your brand identity? We can help!
Essentials of Automations: Exploring Attributes & Automation ParametersSafe Software
Building automations in FME Flow can save time, money, and help businesses scale by eliminating data silos and providing data to stakeholders in real-time. One essential component to orchestrating complex automations is the use of attributes & automation parameters (both formerly known as “keys”). In fact, it’s unlikely you’ll ever build an Automation without using these components, but what exactly are they?
Attributes & automation parameters enable the automation author to pass data values from one automation component to the next. During this webinar, our FME Flow Specialists will cover leveraging the three types of these output attributes & parameters in FME Flow: Event, Custom, and Automation. As a bonus, they’ll also be making use of the Split-Merge Block functionality.
You’ll leave this webinar with a better understanding of how to maximize the potential of automations by making use of attributes & automation parameters, with the ultimate goal of setting your enterprise integration workflows up on autopilot.
The Department of Veteran Affairs (VA) invited Taylor Paschal, Knowledge & Information Management Consultant at Enterprise Knowledge, to speak at a Knowledge Management Lunch and Learn hosted on June 12, 2024. All Office of Administration staff were invited to attend and received professional development credit for participating in the voluntary event.
The objectives of the Lunch and Learn presentation were to:
- Review what KM ‘is’ and ‘isn’t’
- Understand the value of KM and the benefits of engaging
- Define and reflect on your “what’s in it for me?”
- Share actionable ways you can participate in Knowledge - - Capture & Transfer
How information systems are built or acquired puts information, which is what they should be about, in a secondary place. Our language adapted accordingly, and we no longer talk about information systems but applications. Applications evolved in a way to break data into diverse fragments, tightly coupled with applications and expensive to integrate. The result is technical debt, which is re-paid by taking even bigger "loans", resulting in an ever-increasing technical debt. Software engineering and procurement practices work in sync with market forces to maintain this trend. This talk demonstrates how natural this situation is. The question is: can something be done to reverse the trend?
TAROT2013 Testing School - Myra Cohen presentation
1. Sampling, Re-use and Incremental Testing in
Software Product Lines
Myra B. Cohen
myra@cse.unl.edu
University of Nebraska-Lincoln
2. Agenda
SPL System Testing: the combinatorics
SPL Integration Testing
Summary
Continuous Test Suite Augmentation
Introduction to Software Product Lines
4. Software Product Lines
• A software product line (SPL) is a set of programs that share a
significant common (managed) set of features satisfying a
market segment or mission developed from a common set of
core assets in a prescribed way
Product 1
Commonality
Variability
..
.Product 2 Product n
Core Assets
5. Feature Oriented Domain
Analysis
• SEI FODA Project in Late 1980s
• Identified features (variability) as the key to software
product lines
• Identified the need for artifact-independent modeling of the
features in an SPL
•
• Introduced the feature diagram which led to what we know
today a the feature model
6. Feature Models
• Representation of the features and
dependencies/constraints in an SPL
• Defines what is (and is not) allowable in
the SPL
• Can be written as a logic equation and can
use SAT solvers to reason about model
• Several different notations for feature
models
7. Software Product Line
Engineering
• Consists of two parts:
– Domain engineering
• Determining what the re-usable components/
architecture is for the SPL
• Defines the commonality and variability of the
SPL
– Application engineering
• Deriving products from the platform created in
the domain engineering step
• Makes use of re-use and exploits variability/
commonality
8. Domain Testing
• Validate and verify reusable
components
• No running application exists at this
time - only components
• This differs from system/application
testing where the entire system is tested
as a unit
10. Examples of Variability
• Electric bulb can be lit or unlit
• Software applications support different
languages
• A triple band mobile phone supports
three different network standards
• Can load Choco vs. MiniSat to solve
constraints
13. V V V
16MC 8MC BW
V V
GV TV
V V
2MP 1MP
Viewer
Type
VP
Camera
Type
VP
Phone
VP
V
Display
V
Email
Viewer
V
Camera
V
Video
Camera
V
Video
Ringtones
requires_v_vp
requires_v_vp requires_v_vp
Mobile Phone SPL
Display
Type
VP
14. V V V
16MC 8MC BW
V V
GV TV
V V
2MP 1MP
Viewer
Type
VP
Camera
Type
VP
Phone
VP
V
Display
V
Email
Viewer
V
Camera
V
Video
Camera
V
Video
Ringtones
requires_v_vp
requires_v_vp requires_v_vp
Mobile Phone SPL
Display
Type
VP
15. V V V
16MC 8MC BW
V V
GV TV
V V
2MP 1MP
Viewer
Type
VP
Camera
Type
VP
Phone
VP
V
Display
V
Email
Viewer
V
Camera
V
Video
Camera
V
Video
Ringtones
requires_v_vp
requires_v_vp requires_v_vp
Mobile Phone SPL
Display
Type
VP
16. V V V
16MC 8MC BW
V V
GV TV
V V
2MP 1MP
Viewer
Type
VP
Camera
Type
VP
Phone
VP
V
Display
V
Email
Viewer
V
Camera
V
Video
Camera
V
Video
Ringtones
requires_v_vp
requires_v_vp requires_v_vp
Mobile Phone SPL
Display
Type
VP
17. V V V
16MC 8MC BW
V V
GV TV
V V
2MP 1MP
Viewer
Type
VP
Camera
Type
VP
Phone
VP
V
Display
V
Email
Viewer
V
Camera
V
Video
Camera
V
Video
Ringtones
requires_v_vp
requires_v_vp requires_v_vp
Mobile Phone SPL
Display
Type
VP
18. V V V
16MC 8MC BW
V V
GV TV
V V
2MP 1MP
Viewer
Type
VP
Camera
Type
VP
Phone
VP
V
Display
V
Email
Viewer
V
Camera
V
Video
Camera
V
Video
Ringtones
requires_v_vp
requires_v_vp requires_v_vp
Mobile Phone SPL
Display
Type
VP
19. V V V
16MC 8MC BW
V V
GV TV
V V
2MP 1MP
Viewer
Type
VP
Camera
Type
VP
Phone
VP
V
Display
V
Email
Viewer
V
Camera
V
Video
Camera
V
Video
Ringtones
requires_v_vp
requires_v_vp requires_v_vp
Mobile Phone SPL
Display
Type
VP
20. V V V
16MC 8MC BW
V V
GV TV
V V
2MP 1MP
Viewer
Type
VP
Camera
Type
VP
Phone
VP
V
Display
V
Email
Viewer
V
Camera
V
Video
Camera
V
Video
Ringtones
requires_v_vp
requires_v_vp requires_v_vp
Mobile Phone SPL
Display
Type
VP
21. Testing a Product Line
• SPL engineering decreases time to market
• With shorter life cycles, the percentage of
time spent in testing has increased – the new
bottleneck
• Much of the current work on testing SPLs
focuses on testing individual instances and
reuse of specific test cases
22. V V V
16MC 8MC BW
V V
GV TV
V V
2MP 1MP
Viewer
Type
VP
Camera
Type
VP
Phone
VP
V
Display
V
Email
Viewer
V V
Camera
Video
Camera
V
Video
Ringtones
requires_v_vp
requires_v_vp requires_v_vp
Validating at the Unit Level
Display
Type
VP
23. V V V
16MC 8MC BW
Validating at the Unit Level
Display
Type
VP
V
16MC
24. V V V
16MC 8MC BW
Validating at the Unit Level
Display
Type
VP
V
16MC
V
8MC
25. V V V
16MC 8MC BW
Validating at the Unit Level
Display
Type
VP
V
16MC
V
8MC
V
BW
Focus is on reuse of artifacts during testing
Views testing from the feature level rather
than the system level
26. V V V
16MC 8MC BW
V V
GV TV
V V
2MP 1MP
Viewer
Type
VP
Camera
Type
VP
Phone
VP
V
Display
V
Email
Viewer
V
Camera
V
Video
Camera
V
Video
Ringtones
requires_v_vp
requires_v_vp requires_v_vp
System Level Testing
Display
Type
VP
27. V V V
16MC 8MC BW
V V
GV TV
V V
2MP 1MP
Viewer
Type
VP
Camera
Type
VP
Phone
VP
V
Display
V
Email
Viewer
V
Camera
V
Video
Camera
V
Video
Ringtones
requires_v_vp
requires_v_vp requires_v_vp
System Level Testing
Viewer
Type
VP
Test Case: Open email in HTML format
28. V V V
16MC 8MC BW
V V
GV TV
V V
2MP 1MP
Viewer
Type
VP
Camera
Type
VP
Phone
VP
V
Display
V
Email
Viewer
V
Camera
V
Video
Camera
V
Video
Ringtones
requires_v_vp
requires_v_vp requires_v_vp
System Level Testing
Display
Type
VP
Test Case: Open email in HTML format
29. V V V
16MC 8MC BW
V V
GV TV
V V
2MP 1MP
Viewer
Type
VP
Camera
Type
VP
Phone
VP
V
Display
V
Email
Viewer
V
Camera
V
Video
Camera
V
Video
Ringtones
requires_v_vp
requires_v_vp requires_v_vp
Feature Interactions
Viewer
Type
VP
Failure: caused by interaction of data passed
between 16MC display and textual viewer
31. Real Interaction Failure
• NASA Detected failed communications on
two independent channels
• Determined that the rover was continually
re-booting – risked running out of energy
and/or overheating
• Fault caused by assumptions made
between two different features: (NASA component
expected third party file system to deallocate memory)
– when one feature was removed the reboots
stopped
32. V V V
16MC 8MC BW
V V
GV TV
V V
2MP 1MP
Viewer
Type
VP
Camera
Type
VP
Phone
VP
V
Display
V
Email
Viewer
V
Camera
V
Video
Camera
V
Video
Ringtones
requires_v_vp
requires_v_vp requires_v_vp
Possible Interactions
Display
Type
VP
33. Fault Map of Configurations
V4
F1 F2 F3 F4 F5 F6 F7 F8 F9 F10 F11 F12 F13 F14
C0
C10
C20
C30
C40
C50
C60
F1 F2 F3 F4 F5
C0
C10
C20
C30
C40
C50
C60
Faults
C
o
n
f
I
g
u
r
a
t
I
o
n
s
34. V V V
16MC 8MC BW
V V
GV TV
V V
2MP 1MP
Viewer
Type
VP
Camera
Type
VP
Phone
VP
V
Display
V
Email
Viewer
V
Camera
V
Video
Camera
V
Video
Ringtones
requires_v_vp
requires_v_vp requires_v_vp
Testing the Interaction Space
Viewer
Type
VP
Number of possible interactions grows
exponentially with number of variation points
35. Some Other Approaches to
SPL Testing
• Extended use cases for variability [Bertolino et al., SPFE 2003]
• Integration testing a SPL based on UML activity
diagrams [Reis et al., FASE 2007]
• Generate test cases incrementally for a SPL
[Uzuncaova et al., ISSRE 2008]
• Applying test cases for a subset of products only
[Kim et al., ASE 2010]
• Applying monitoring properties for a subset of
products only [Kim et al., RV 2010]
36. Some SPL Resources
• http://splot-research.org/
• http://www.isa.us.es/fama/
• http://sir.unl.edu
37. Agenda
SPL System Testing: the combinatorics
SPL Integration Testing
Summary
Continuous Test Suite Augmentation
Introduction to Software Product Lines
38. A Combinatorial Problem
Display Email Viewer Camera Video
Camera
Video
Ringtones
16 Million Colors Graphical 2 Megapixels Yes Yes
8 Million Colors Text 1 Megapixel No No
Black and White None None
39. A Combinatorial Problem
Display Email Viewer Camera Video
Camera
Video
Ringtones
16 Million Colors Graphical 2 Megapixels Yes Yes
8 Million Colors Text 1 Megapixel No No
Black and White None None
3
40. A Combinatorial Problem
Display Email Viewer Camera Video
Camera
Video
Ringtones
16 Million Colors Graphical 2 Megapixels Yes Yes
8 Million Colors Text 1 Megapixel No No
Black and White None None
3 x 3
41. A Combinatorial Problem
Display Email Viewer Camera Video
Camera
Video
Ringtones
16 Million Colors Graphical 2 Megapixels Yes Yes
8 Million Colors Text 1 Megapixel No No
Black and White None None
• 33x22=108 feasible SPL instances
3 x 3 x 3 x 2 x 2
42. A Combinatorial Problem
• 33x22=108 feasible SPL instances
• 10 features each with 5 values = 510 or
9,765,625 possible instances
Display Email Viewer Camera Video
Camera
Video
Ringtones
16 Million Colors Graphical 2 Megapixels Yes Yes
8 Million Colors Text 1 Megapixel No No
Black and White None None
43. A Combinatorial Problem
• 33x22=108 feasible SPL instances
• 10 features each with 5 values = 510 or
9,765,625 possible instances
• 4 hours to run test suite = approximately 4,459 years
to test
GCC optimizer: configuration space ~1061
Display Email Viewer Camera Video
Camera
Video
Ringtones
16 Million Colors Graphical 2 Megapixels Yes Yes
8 Million Colors Text 1 Megapixel No No
Black and White None None
44. Combinatorial Interaction Testing
(CIT)
• Test all pairs or t-way combinations of
feature-values, where t is a defined
strength of testing
• Uses a mathematical structure called a
covering array
45. Display Email Viewer Camera Video V. Ringtones
1 16 Million Colors None 1 Megapixel Yes Yes
2 8 Million Colors Text None No No
3 16 Million Colors Text 2 Megapixels No Yes
4 Black and White None None No Yes
5 8 Million Colors None 2 Megapixels Yes No
6 16 Million Colors Graphical None Yes No
7 Black and White Text 1 Megapixel Yes No
8 8 Million Colors Graphical 1 Megapixel No Yes
9 Black and White Graphical 2 Megapixels Yes Yes
2-way CIT
46. Display Email Viewer Camera Video V. Ringtones
1 16 Million Colors None 1 Megapixel Yes Yes
2 8 Million Colors Text None No No
3 16 Million Colors Text 2 Megapixels No Yes
4 Black and White None None No Yes
5 8 Million Colors None 2 Megapixels Yes No
6 16 Million Colors Graphical None Yes No
7 Black and White Text 1 Megapixel Yes No
8 8 Million Colors Graphical 1 Megapixel No Yes
9 Black and White Graphical 2 Megapixels Yes Yes
2-way CIT
47. Display Email Viewer Camera Video V. Ringtones
1 16 Million Colors None 1 Megapixel Yes Yes
2 8 Million Colors Text None No No
3 16 Million Colors Text 2 Megapixels No Yes
4 Black and White None None No Yes
5 8 Million Colors None 2 Megapixels Yes No
6 16 Million Colors Graphical None Yes No
7 Black and White Text 1 Megapixel Yes No
8 8 Million Colors Graphical 1 Megapixel No Yes
9 Black and White Graphical 2 Megapixels Yes Yes
2-way CIT
48. Combinatorial Interaction Testing
(CIT)
• Based on statistical design of experiments (DOE)
– Manufacturing
– Drug test interactions
– Chemical interactions
• For software testing
– Mandl – compiler testing
– Brownlie, Prowse, Phadke – OATS system
– D. Cohen, Dalal, Fredman, Patton, Parelius –
AETG
– Williams, Probert – network node interfaces
– Yilmaz, Cohen, Porter- ACE/TAO
49. Empirical Results on CIT
for System Inputs
• CIT has been well studied on testing input
parameters for programs
• [Brownlie, Prowse, Phadke 92], [Burr,Young
98],[Burroughs, Jain, Erickson 94] [Cohen,
Dalal, Fredman, Patton 97], [Dalal, Jain,
Karunanithi, Leaton, Lott 98], [Dunietz, Erlich,
Szablak, Mallows, Iannino 97], [Mandl 85]
$ sort -m -c input.txt
50. CIT for Configurable
Systems
• Some studies on configurable
systems
• [Kuhn, Wallace,Gallo 04]
– Medical devices, web browsers, data mgt, http server
• [Yilmaz, Cohen, Porter 04,06]
– Ace/TAO Corba middleware
• [Qu, Cohen, Rothermel 08]
51. CIT vs. Default
Configuration of vim in SIRFaultDetection(%)
020406080100
v2 v3 v4 v5 v6 v7
Version default configuration
52. How do we Generate the
Samples?
• Greedy approaches
– AETG, IPOG
– Search-based (simulated annealing, GAs)
• Use of constraint solvers
55. Meta-heuristic Algorithms
• Provide mechanisms to escape local
optima
• Sometimes the algorithm can accept a
worse choice in the hope we will find a
better route to a good solution
59. • Meant to simulate the coolng of metal
• Physical annealing is a process of slowly cooling
a metal, in baths of decreasing temperatures.
• At each stage the metal molecules are allowed
to come to an equilibrium. Then the bath is
lowered to the next temperature.
• If the initial temperature is not high enough or if
the cooling is done too quickly the metal will be
brittle.
Simulated Annealing
60. • The basic strategy is the same as a hill climb,
but the meta-heuristic creates a gentle decrease
in the probability for making a bad move.
• At the start of our hill climb we often make a lot
of bad moves.
• By the time we are close to an optimal solution
we are almost in a pure hill climb (the
probability drops to almost zero)
Simulated Annealing (cont.)
61. • Start with a solution S
• Select a new solution S’ that is a neighbor of S
• If S’-S ≤ 0 then S’ is accepted
• Otherwise
– Accept S’ with a probability
Where T = temperature and B is the Boltzmann
constant (this is a distribution which we won’t
actually use for simulating the process)
Simulated Annealing
€
e
s−s'
kBT
63. Lower bound L
Binary search for “best” solution
Upper bound U
(L+U)/2
Finding “N”
64. The Real System
Constraints on Valid Configurations:
__________________________________________________________________________________________________
(1) Graphical email viewer requires color display
(2) 2 Megapixel camera requires a color display
(3) Graphical email viewer not supported with 2 Megapixel camera
(4) 8 Million color display does not support a 2 Megapixel camera
(5) Video camera requires a camera and a color display
(6) Video ringtones cannot occur with No video camera
(7) The combination of 16 Million colors, Text and
2 Megapixel camera will not be supported
65. Display Email Viewer Camera Video V. Ringtones
1 16 Million Colors None 1 Megapixel Yes Yes
2 8 Million Colors Text None No No
3 16 Million Colors Text 2 Megapixels No Yes
4 Black and White None None No Yes
5 8 Million Colors None 2 Megapixels Yes No
6 16 Million Colors Graphical None Yes No
7 Black and White Text 1 Megapixel Yes No
8 8 Million Colors Graphical 1 Megapixel No Yes
9 Black and White Graphical 2 Megapixels Yes Yes
Constraints
66. Display Email Viewer Camera Video V. Ringtones
1 16 Million Colors None 1 Megapixel Yes Yes
2 8 Million Colors Text None No No
3 16 Million Colors Text 2 Megapixels No Yes
4 Black and White None None No Yes
5 8 Million Colors None 2 Megapixels Yes No
6 16 Million Colors Graphical None Yes No
7 Black and White Text 1 Megapixel Yes No
8 8 Million Colors Graphical 1 Megapixel No Yes
9 Black and White Graphical 2 Megapixels Yes Yes
Constraints
“Video Ringtones cannot occur
without video camera
67. Display Email Viewer Camera Video V. Ringtones
1 16 Million Colors None 1 Megapixel Yes Yes
2 8 Million Colors Text None No No
3 16 Million Colors Text 2 Megapixels No Yes
4 Black and White None None No Yes
5 8 Million Colors None 2 Megapixels Yes No
6 16 Million Colors Graphical None Yes No
7 Black and White Text 1 Megapixel Yes No
8 8 Million Colors Graphical 1 Megapixel No Yes
9 Black and White Graphical 2 Megapixels Yes Yes
Constraints
“Video Ringtones cannot occur
without video camera
68. Display Email Viewer Camera Video
Camera
Video
Ringtones
1 16 Million Colors None 1 Megapixel Yes Yes
2 8 Million Colors Text None No No
3 16 Million Colors Text 2 Megapixels No Yes
4 Black and White None None No Yes
5 8 Million Colors None 2 Megapixels Yes No
6 16 Million Colors Graphical None Yes No
7 Black and White Text 1 Megapixel Yes No
8 8 Million Colors Graphical 1 Megapixel No Yes
9 Black and White Graphical 2 Megapixels Yes Yes
Constraints:
(1) Graphical email viewer requires color display
(2) 2 Megapixel Camera requires color display
(3) Graphical email viewer not supported with the 2 Megapixel
camera
Decrease Sample Size
69. Display Email Viewer Camera Video
Camera
Video
Ringtones
1 16 Million Colors None 2 Megapixels No No
2 16 Million Colors Graphical 1 Megapixel No No
3 8 Million Colors None 1 Megapixel Yes Yes
4 16 Million Colors Graphical None Yes Yes
5 16 Million Colors Text 1 Megapixel Yes No
6 Black and White None 1 Megapixel Yes Yes
7 8 Million Colors Graphical 2 Megapixels Yes Yes
8 Black and White Text 2 Megapixels No Yes
9 8 Million Colors Text None No No
10 Black and White None None No No
Constraints:
_____________________________________________________________________
(1) Graphical email viewer requires color display
Increase Sample Size
70. Display Email Viewer Camera Video
Camera
Video
Ringtones
1 16 Million Colors None 2 Megapixels No No
2 16 Million Colors Graphical 1 Megapixel No No
3 8 Million Colors None 1 Megapixel Yes Yes
4 16 Million Colors Graphical None Yes Yes
5 16 Million Colors Text 1 Megapixel Yes No
6 Black and White None 1 Megapixel Yes Yes
7 8 Million Colors Graphical 2 Megapixels Yes Yes
8 Black and White Text 2 Megapixels No Yes
9 8 Million Colors Text None No No
10 Black and White None None No No
Constraints:
_____________________________________________________________________
(1) Graphical email viewer requires color display
Increase Sample Size
71. Display Email Viewer Camera Video
Camera
Video
Ringtones
1 16 Million Colors None 2 Megapixels No No
2 16 Million Colors Graphical 1 Megapixel No No
3 8 Million Colors None 1 Megapixel Yes Yes
4 16 Million Colors Graphical None Yes Yes
5 16 Million Colors Text 1 Megapixel Yes No
6 Black and White None 1 Megapixel Yes Yes
7 8 Million Colors Graphical 2 Megapixels Yes Yes
8 Black and White Text 2 Megapixels No Yes
9 8 Million Colors Text None No No
10 Black and White None None No No
Constraints:
_____________________________________________________________________
(1) Graphical email viewer requires color display
Increase Sample Size
72. Display Email Viewer Camera Video
Camera
Video
Ringtones
1 16 Million Colors None 2 Megapixels No No
2 16 Million Colors Graphical 1 Megapixel No No
3 8 Million Colors None 1 Megapixel Yes Yes
4 16 Million Colors Graphical None Yes Yes
5 16 Million Colors Text 1 Megapixel Yes No
6 Black and White None 1 Megapixel Yes Yes
7 8 Million Colors Graphical 2 Megapixels Yes Yes
8 Black and White Text 2 Megapixels No Yes
9 8 Million Colors Text None No No
10 Black and White None None No No
Constraints:
_____________________________________________________________________
(1) Graphical email viewer requires color display
Increase Sample Size
74. Greedy Worked Well
• Extended both greedy and meta-
heuristic search algorithms to find
constrained CIT samples
• Used off the shelf SAT solver
• Mine information learned during
Boolean propagation to improve
performance of greedy algorithm
M.B. Cohen, M.B. Dwyer and J. Shi, Constructing interaction test suites for
highly-configurable systems in the presence of constraints: a greedy approach,
IEEE Transactions on Software Engineering , 34(5), 2008, pp. 633-650.
75. Limitations
• Greedy algorithms tend to:
• Finish quickly
• Handle constraints well
• Meta-heuristic algorithms tend to:
• Find smaller samples
• Handle constraints poorly
• Software product lines may have long test times per
instance and many dependencies:
• Want a meta-heuristic search algorithm that builds small
samples quickly
76. Reformulated Simulated
Annealing CIT Algorithm
• Modified:
– Neighborhood of search
• Narrows as search progresses
– Global strategy to find minimal size of array
– A set of minor modifications
• B.J. Garvin, M.B. Cohen, and M.B. Dwyer, Evaluating Improvements to a
Meta-Heuristic Search for Constrained Interaction Testing, Empirical Software
Engineering (EMSE), 16(1), 2011, pp.61-102.
• CASA tool: http://cse.unl.edu/~citportal/tools/casa/
77. Major Modifications
• Original implementation uses two
phases:
(1) Finding minimal size of sample
(2) Finding a valid sample within that size
• Modifications:
• New search space (neighborhood)
• Improved method for finding size (global)
78. Reduces infeasible search paths (dotted lines)
Original Search Space may lead to
many infeasible solutions
New Search Space provides a more direct
path to feasible solutions
Modified Search Space
(CASA)1
1. Available at: http://www.cse.unl.edu/citportal/tools/casa/
79. Reduces infeasible search paths (dotted lines)
Original Search Space may lead to
many infeasible solutions
New Search Space provides a more direct
path to feasible solutions
Modified Search Space
(CASA)
80. Reduces infeasible search paths (dotted lines)
Original Search Space may lead to
many infeasible solutions
New Search Space provides a more direct
path to feasible solutions
Modified Search Space
(CASA)
82. Case Studies (t=2)
Average Size Average Time Break
Even Point
Greedy Old SA New SA Greedy Old SA New SA Greedy vs.
New SA
SPINs 27.0 20.2 19.4 0.2 414.9 8.6 1.1
SPINv 42.5 33.2 36.8 11.3 4010.7 102.1 15.9
GCC 24.7 19.8 21.1 204.0 36,801.4 1902.0 471.7
Apac 42.6 32.6 32.3 76.4 23,589.1 109.1 3.2
Bugz 21.8 16.0 16.2 1.9 58.0 9.1 1.3
83. Case Studies (t=2)
Average Size Average Time Break
Even Point
Greedy Old SA New SA Greedy Old SA New SA Greedy vs.
New SA
SPINs 27.0 20.2 19.4 0.2 414.9 8.6 1.1
SPINv 42.5 33.2 36.8 11.3 4010.7 102.1 15.9
GCC 24.7 19.8 21.1 204.0 36,801.4 1902.0 471.7
Apac 42.6 32.6 32.3 76.4 23,589.1 109.1 3.2
Bugz 21.8 16.0 16.2 1.9 58.0 9.1 1.3
84. Case Studies (t=2)
Average Size Average Time Break
Even Point
Greedy Old SA New SA Greedy Old SA New SA Greedy vs.
New SA
SPINs 27.0 20.2 19.4 0.2 414.9 8.6 1.1
SPINv 42.5 33.2 36.8 11.3 4010.7 102.1 15.9
GCC 24.7 19.8 21.1 204.0 36,801.4 1902.0 471.7
Apac 42.6 32.6 32.3 76.4 23,589.1 109.1 3.2
Bugz 21.8 16.0 16.2 1.9 58.0 9.1 1.3
85. Case Studies (t=2)
Average Size Average Time Break
Even Point
Greedy Old SA New SA Greedy Old SA New SA Greedy vs.
New SA
SPINs 27.0 20.2 19.4 0.2 414.9 8.6 1.1
SPINv 42.5 33.2 36.8 11.3 4010.7 102.1 15.9
GCC 24.7 19.8 21.1 204.0 36,801.4 1902.0 471.7
Apac 42.6 32.6 32.3 76.4 23,589.1 109.1 3.2
Bugz 21.8 16.0 16.2 1.9 58.0 9.1 1.3
86. Case Studies (t=2)
Average Size Average Time Break
Even Point
Greedy Old SA New SA Greedy Old SA New SA Greedy vs.
New SA
SPINs 27.0 20.2 19.4 0.2 414.9 8.6 1.1
SPINv 42.5 33.2 36.8 11.3 4010.7 102.1 15.9
GCC 24.7 19.8 21.1 204.0 36,801.4 1902.0 471.7
Apac 42.6 32.6 32.3 76.4 23,589.1 109.1 3.2
Bugz 21.8 16.0 16.2 1.9 58.0 9.1 1.3
87. Case Studies (t=2)
Average Size Average Time Break
Even Point
Greedy Old SA New SA Greedy Old SA New SA Greedy vs.
New SA
SPINs 27.0 20.2 19.4 0.2 414.9 8.6 1.1
SPINv 42.5 33.2 36.8 11.3 4010.7 102.1 15.9
GCC 24.7 19.8 21.1 204.0 36,801.4 1902.0 471.7
Apac 42.6 32.6 32.3 76.4 23,589.1 109.1 3.2
Bugz 21.8 16.0 16.2 1.9 58.0 9.1 1.3
88. Going Beyond CIT
• CIT is a system testing technique:
– Need to instantiate complete products
– Many interactions are low order
• CIT is black box:
– Tests may or may not actually trigger
interactions between features
– No consideration of data/control flow
directions
89. Directed vs. Undirected
Interactions
Display Email Viewer Camera Video V. Ringtones
1 16 Million Colors None 1 Megapixel Yes Yes
2 8 Million Colors Text None No No
3 16 Million Colors Text 2 Megapixels No Yes
4 Black and White None None No Yes
5 8 Million Colors None 2 Megapixels Yes No
6 16 Million Colors Graphical None Yes No
7 Black and White Text 1 Megapixel Yes No
8 8 Million Colors Graphical 1 Megapixel No Yes
9 Black and White Graphical 2 Megapixels Yes Yes
90. Directed vs. Undirected
Interactions
Display Email Viewer Camera Video V. Ringtones
1 16 Million Colors None 1 Megapixel Yes Yes
2 8 Million Colors Text None No No
3 16 Million Colors Text 2 Megapixels No Yes
4 Black and White None None No Yes
5 8 Million Colors None 2 Megapixels Yes No
6 16 Million Colors Graphical None Yes No
7 Black and White Text 1 Megapixel Yes No
8 8 Million Colors Graphical 1 Megapixel No Yes
9 Black and White Graphical 2 Megapixels Yes Yes
2 ways to test this interaction:
8 million colors à Text
Text à 8 million colors
94. Agenda
SPL System Testing: the combinatorics
SPL Integration Testing
Summary
Continuous Test Suite Augmentation
Introduction to Software Product Lines
95. Our Idea
• Test at the integration level:
• We can test partial products
• Incrementally increase strength of interactions
• Re-use results as we test
• Reduce the interaction space:
• Identify only feasible interactions
• Integrate features using the analysis of feasible
interactions
• Generate tests to exercise these interactions
(future)
96. Approach
s() {!
…!
v();!
if (c) {!
…!
}!
w();!
…!
}!
f1(){!
…!
}!
f2(){!
…!
}!
f3() !
f4() !
J. Shi, Myra B. Cohen and Matthew B. Dwyer, Integration Testing of Software
Product Lines Using Compositional Symbolic Execution, International Conference on
Fundamental Approaches to Software Engineering (FASE), March 2012, pp. 270-284.
97. Approach
s() {!
…!
v();!
if (c) {!
…!
}!
w();!
…!
}!
f1(){!
…!
}!
f2(){!
…!
}!
f3() !
f4() !
CFG
PDG
Code: Program Dependency
and Control Flow Graph
98. Approach
s() {!
…!
v();!
if (c) {!
…!
}!
w();!
…!
}!
f1(){!
…!
}!
f2(){!
…!
}!
f3() !
f4() !
Interaction trees
CFG
PDG
Feature Model: Defines possible
interaction space
111. Testing Method
• Use bounded symbolic execution
• Obtain summaries from individual
features
• Use interaction trees to guide
composition of all t-feature summaries
112. Symbolic Summaries
• We have three outcomes:
1. Complete execution of a path (passing
behavior)
2. Exception (failing behavior)
– may or may not be feasible to reach in full system
3. Depth bound is reach (unknown behavior)
In this work we only accumulate type 1.
113. Results of (Composed)
Summaries
• Constraints can be collected for each region to
perform test generation (in progress)
sum(f1,f2,f4) = {(f1.x0,f4.ret=f1.x+1), …)
complete
exception
Time out
complete
exception
Time out
Feature 1 Feature 2
117. Case Study
We performed a case study on 2 software
product lines to answer these questions:
• RQ1: What is the reduction from our dependency
analysis on the number of interactions that should be
tested in an SPL?
• RQ2: What is the difference in time between using
our compositional symbolic technique versus a
traditional directed technique?
121. Method
RQ1:
• Compared number of (directed/undirected)
interactions
RQ2:
• Compared directed SE vs. incremental
compositional SE
122. Method
• Tools:
– Adapted the IFA in Soot
– Direct SE – use Java PathFinder
– Compositional approach – built a composer
• Depth:
– 20*t
– Explicit for Direct SE
– Implicit for compositional
135. RQ1: Summary
• We see a reduction of over 50% in
directed and over 75% in undirected
interactions when we use the interaction
trees
136. Subject t Direct
SE
Comp
SE
SCARI 1 6.75 6.75
2 14.48 9.63
3 17.67 10.06
4 36.09 10.93
5 35.87 11.70
GPL 1 41.77 41.77
2 67.25 56.28
3 184.76 82.00
4 727.34 216.63
5 3887.23 965.92
Compositional vs.
Directed(seconds)
137. Subject t Direct
SE
Comp
SE
SCARI 1 6.75 6.75
2 14.48 9.63
3 17.67 10.06
4 36.09 10.93
5 35.87 11.70
GPL 1 41.77 41.77
2 67.25 56.28
3 184.76 82.00
4 727.34 216.63
5 3887.23 965.92
Compositional vs.
Directed(seconds)
138. Subject t Direct
SE
Comp
SE
SCARI 1 6.75 6.75
2 14.48 9.63
3 17.67 10.06
4 36.09 10.93
5 35.87 11.70
GPL 1 41.77 41.77
2 67.25 56.28
3 184.76 82.00
4 727.34 216.63
5 3887.23 965.92
Compositional vs.
Directed(seconds)
139. Subject t Direct
SE
Comp
SE
SCARI 1 6.75 6.75
2 14.48 9.63
3 17.67 10.06
4 36.09 10.93
5 35.87 11.70
GPL 1 41.77 41.77
2 67.25 56.28
3 184.76 82.00
4 727.34 216.63
5 3887.23 965.92
Compositional vs.
Directed(seconds)
140. Subject t Direct
SE
Comp
SE
SCARI 1 6.75 6.75
2 14.48 9.63
3 17.67 10.06
4 36.09 10.93
5 35.87 11.70
GPL 1 41.77 41.77
2 67.25 56.28
3 184.76 82.00
4 727.34 216.63
5 3887.23 965.92
Compositional vs.
Directed(seconds)
Same time
141. Subject t Direct
SE
Our
Tech
SCARI 1 6.75 6.75
2 14.48 9.63
3 17.67 10.06
4 36.09 10.93
5 35.87 11.70
GPL 1 41.77 41.77
2 67.25 56.28
3 184.76 82.00
4 727.34 216.63
5 3887.23 965.92
1/3 of the time
1/4 of the time
Compositional vs.
Directed(seconds)
144. RQ2: Summary
• As we increase the interaction strength
we see a greater reduction in time when
using compositional symbolic execution
• We reduce the time by a factor of 4 in
the best case
• We believe this will allow for greater
scalability and testing of higher
strengths
145. Summary
• We developed an incremental
compositional symbolic execution
technique for integration testing of SPLs
• Leverages dependency analysis to
reduce interactions by more than 50%
• Compositional symbolic execution
reduces time by as much as 75%
146. Future Directions
• Use summaries for test generation to
execute specific interaction
• Characterize areas of behavior to drive
further test generation
• Extend to other types of SPLs
147. Agenda
SPL System Testing: the combinatorics
SPL Integration Testing
Summary
Continuous Test Suite Augmentation
Introduction to Software Product Lines
148. COntinuous TEst Suite
Augmentation
• CONTESA
Z. Xu, M. B. Cohen, W. Motycka, G. Rothermel, Continuous Test Suite Augmentation
in Software Product Lines, International Software Product Line Conference (SPLC),
August, 2013, (to appear).
149. Approaches to Testing/
Analyzing SPLS1
• Feature based
– Focusing on individual features
• Product based
– Focusing on products
• Family based
– Testing from family perspective
1. T. Thum, S. Apel, C. Kastner, M. Kuhlemann, I. Schaefer, and G. Saake. Analysis
strategies for software product lines. Technical report, Faukulat fur Informatik,
Otto-von-Guericke-Univeritat, April 2012.
150. CONTESA
• Hybrid family/product based approach
• Aims to cover all code in family by
selecting products to test
• Leverages ideas/techniques from
regression testing
157. Test Cases Change Coverage
Tests
We need to add tests to the test
suite
158. Code-based View
E
int foo (int x, int y)
P1: if(x0)
S1:return
x;
S3:return
x;
P3: if(x3)
X
S2:return
x+2;
S4:return
x+y;
P2: if(xy)
T
T
T
F
F
F
Test Suite:
t1=(x = 2, y = 2) t2=(x = 4, y = 4)
t3=(x = 1, y = 0) t4=(x = 4, y = 3)
t5=(x = −1, y = 0)
159. Code-based View
E
int foo (int x, int y)
P1: if(x0)
S1:return
x;
S3:return
x;
P3: if(x3)
X
S2:return
x+2;
S4:return
x+y;
P2: if(xy)
T
T
T
F
F
F
Test Suite:
t1=(x = 2, y = 2) t2=(x = 4, y = 4)
t3=(x = 1, y = 0) t4=(x = 4, y = 3)
t5=(x = −1, y = 0)
160. Code-based View
E
int foo (int x, int y)
P1: if(x0)
S1:return
x;
S3:return
x;
P3: if(x3)
X
S2:return
x+2;
S4:return
x+y;
P2: if(xy)
T
T
T
F
F
F
Test Suite:
t1=(x = 2, y = 2) t2=(x = 4, y = 4)
t3=(x = 1, y = 0) t4=(x = 4, y = 3)
t5=(x = −1, y = 0)
161. Code-based View
E
int foo (int x, int y)
P1: if(x0)
S1:return
x;
S3:return
x;
P3: if(x3)
X
S2:return
x+2;
S4:return
x+y;
P2: if(xy)
T
T
T
F
F
F
Test Suite:
t1=(x = 2, y = 2) t2=(x = 4, y = 4)
t3=(x = 1, y = 0) t4=(x = 4, y = 3)
t5=(x = −1, y = 0)
162. Code-based View
E
int foo (int x, int y)
P1: if(x0)
S1:return
x;
S3:return
x;
P3: if(x3)
X
S2:return
x+2;
S4:return
x+y;
P2: if(xy)
T
T
T
F
F
F
Test Suite:
t1=(x = 2, y = 2) t2=(x = 4, y = 4)
t3=(x = 1, y = 0) t4=(x = 4, y = 3)
t5=(x = −1, y = 0)
164. Implication
E
int foo (int x, int y)
P1: if(x0)
S1:return
x;
S3:return
x;
P3: if(x3)
X
S2:return
x+2;
S4:return
x+y;
P2: if(xy)
T
T
T
F
F
F
E
int foo’ (int x, int y)
P1: if(x0)
S1:return
x;
S3:return
x;
P3: if(x3)
X
S2:return
x+2;
S4:return
x+y;
P2:if(x=y)
T
T
T
F
F
F
167. 1. Identify affected elements (portions of
the program′ or its specification for
which new tests are needed)
2. Create or guide the creation of tests
that exercise these elements
Test Suite Augmentation
168. Test Suite Augmentation
for SPLs
V1 V2 V3 V4 V5 V6
Non-SPL system evolves
Products of an SPL
A
B C
F
D
E
V V V16MC 8MC BW V VGV TV V V
2MP 1MP
Viewer
Type
V
P
Camera
TypeV
P
Phone
V
P
V
Display
V
Email
Viewer
V
Camera
V
Video
Camera
V
Video
Ringtones
requires_v_vp
requires_v_vp
requires_v_vp
Viewer
Type
V
P
170. CONTESA
• Test a single product
• Use regression impact analysis
(DejaVu) to identify common vs.
variable code between current/next
product
• Generate tests to cover variable code in
the new product (use a GA)
171. Two Orders of Testing
Both select product with largest number of
uncovered branches (variable code)
• Static:
– Determine order at start
• Dynamic:
– Re-calculate after each product tested
172. Case Study
• RQ1: Is continuous test suite augmentation more
effective and efficient than generating test cases
independently for each product?
• RQ2: Does the order used in continuous test suite
augmentation matter in terms of effectiveness and
efficiency?
177. Summary
• Continuous test suite augmentation
uses ideas from regression testing to
drive SPL test generation
• It is a hybrid product and family based
approach
• Improves both effectiveness and
efficiency over an individual product
based approach
178. Agenda
SPL System Testing: the combinatorics
SPL Integration Testing
Summary
Continuous Test Suite Augmentation
Introduction to Software Product Lines
179. Summary of Lecture
• We have seen how combinatorics
impact SPL testing
• We have seen some sampling and re-
use techniques
– System and integration level
• We have seen an augmentation
technique aimed at the entire code-
base
180. Students and Collaborators
on this work
Isis Cabral
Matthew Dwyer
Brady Garvin
Wayne Motycka
Gregg Rothermel
Xiao Qu
Jiangfan Shi
Zhihong Xu
181. Sampling, Re-use and Incremental Testing in
Software Product Lines
Myra B. Cohen
myra@cse.unl.edu
University of Nebraska-Lincoln
This work is supported in part by the National Science Foundation through awards CCF-1161767,
CCF-0747009 and by the Air Force Office of Scientific Research through award FA9550-10-1-0406.