The document discusses automated testing of JavaScript code using Jasmine, PhantomJS, and Jenkins. It begins with an agenda that covers an introduction to the speakers, testing JavaScript code and front-end code without a browser, and continuous integration. It then discusses Jasmine features for writing tests, using PhantomJS to run tests without a browser, and integrating testing into a continuous integration workflow with Jenkins.
DevQA: make your testers happier with Groovy, Spock and Geb (Greach 2014)Alvaro Sanchez-Mariscal
Writing functional tests using Geb in a Grails application is fine for a development team. But when you have QA automation engineers, giving them access to the Grails app might not be the best solution (specially when they belong to a different team).
So the same way DevOps allow developers and sysadmins collaborate together, let’s talk about DevQA, and make them happy using a framework stack powered by Groovy.
Besides above considerations, in this talk I will show a live example on how to setup an independent project for functional tests using Gradle, Groovy, Spock and Geb.
Story Testing Approach for Enterprise Applications using Selenium FrameworkOleksiy Rezchykov
Releasing a big software product frequently on the same high quality level could became an impossible task. Story Testing approach gives a possibility for many teams to work for a same product and release it without putting enormous efforts on testing. Approach is based on the BDD technique, Feature Flags and Selenium.
Automated Acceptance Testing (and tool choice)
Automated acceptance testing has many names: acceptance-test driven development (ATDD), story-test driven development (STDD), agile acceptance testing and, most recently, specification by example. At the heart of all these approaches is to produce business-facing tests which are system tests running end-to-end, picking up regression issues and improving confidence that the code works as required.
In this talk, I will contextualise how each of these approaches share in common a three-tier layering strategy: acceptance criteria, test implementation layer and application driver layer. This is important because applying this approach requires a tool choice and each tool tends to have its own sweet (and blind) spot that is best understood through these layers.
I will first deep dive into sample code across a few tools (Cucumber, Fitnesse, Concordion) to illustrate this layering. I use an example that shows how to decouple the GUI from tests (window driver pattern).
Finally, I will look at some typical client scenarios to examine which tools might best suited because tool choice is not simply a host operating system question (.Net, Java, Ruby).
Continuous Testing Meets the Classroom at Code.orgSauce Labs
Code.org's Brian Jordan, a Software Engineer, takes the audience for a fun tour of Code.org’s continuous, automated testing suite. Brian discusses how Code.org approaches testing throughout the product development cycle, given their unique testing challenges—developing interactive, game-like curriculum for just the types of browsers you’d expect to find in school computer labs—from Internet Explorer 9 to iPads across 40+ languages.
Join us for an overview of behavior-driven development and test automation, which aided in the production of a Visualforce/JavaScript application for an enterprise client. Using Cucumber JVM, Selenium, Jenkins, and Git - the team was able to catch regression errors during development. We'll provide an overview of the solution used and how it worked in a real-world environment.
DevQA: make your testers happier with Groovy, Spock and Geb (Greach 2014)Alvaro Sanchez-Mariscal
Writing functional tests using Geb in a Grails application is fine for a development team. But when you have QA automation engineers, giving them access to the Grails app might not be the best solution (specially when they belong to a different team).
So the same way DevOps allow developers and sysadmins collaborate together, let’s talk about DevQA, and make them happy using a framework stack powered by Groovy.
Besides above considerations, in this talk I will show a live example on how to setup an independent project for functional tests using Gradle, Groovy, Spock and Geb.
Story Testing Approach for Enterprise Applications using Selenium FrameworkOleksiy Rezchykov
Releasing a big software product frequently on the same high quality level could became an impossible task. Story Testing approach gives a possibility for many teams to work for a same product and release it without putting enormous efforts on testing. Approach is based on the BDD technique, Feature Flags and Selenium.
Automated Acceptance Testing (and tool choice)
Automated acceptance testing has many names: acceptance-test driven development (ATDD), story-test driven development (STDD), agile acceptance testing and, most recently, specification by example. At the heart of all these approaches is to produce business-facing tests which are system tests running end-to-end, picking up regression issues and improving confidence that the code works as required.
In this talk, I will contextualise how each of these approaches share in common a three-tier layering strategy: acceptance criteria, test implementation layer and application driver layer. This is important because applying this approach requires a tool choice and each tool tends to have its own sweet (and blind) spot that is best understood through these layers.
I will first deep dive into sample code across a few tools (Cucumber, Fitnesse, Concordion) to illustrate this layering. I use an example that shows how to decouple the GUI from tests (window driver pattern).
Finally, I will look at some typical client scenarios to examine which tools might best suited because tool choice is not simply a host operating system question (.Net, Java, Ruby).
Continuous Testing Meets the Classroom at Code.orgSauce Labs
Code.org's Brian Jordan, a Software Engineer, takes the audience for a fun tour of Code.org’s continuous, automated testing suite. Brian discusses how Code.org approaches testing throughout the product development cycle, given their unique testing challenges—developing interactive, game-like curriculum for just the types of browsers you’d expect to find in school computer labs—from Internet Explorer 9 to iPads across 40+ languages.
Join us for an overview of behavior-driven development and test automation, which aided in the production of a Visualforce/JavaScript application for an enterprise client. Using Cucumber JVM, Selenium, Jenkins, and Git - the team was able to catch regression errors during development. We'll provide an overview of the solution used and how it worked in a real-world environment.
Page-Object pattern is very commonly used when implementing Automation frameworks. However, as the scale of the framework grows, there is a limitation on how much reusability really happens. It inherently becomes very difficult to separate the test intent from the business domain.
I talk about this problem, and the solution I have been using - Business Layer - Page-Object pattern, which has helped me keep my code DRY.
For more details (links to slides, etc.), see my blog: http://goo.gl/biRn4n
Learn how your company can add automated testing for accessibility on all platforms. This presentation covers what Intuit has learned while working towards this goal
A method of communicating between two devices
A software function provided at a network address over the web with the service always on
It has an interface described in a machine-processable format
http://www.qualitestgroup.com/
United Global Soft
We provide QTP/QA Automation Online training by real time experts.
Contact : +91 8099902123
+1-201-710-8393
Mail Id : info@unitedglobalsoft.com
Better Page Object Handling with Loadable Component Pattern - SQA Days 20, Be...Sargis Sargsyan
One of the painful problems in Selenium automated testing is determining whether a HTML page has been loaded. This is especially the case when web application uses a JS heavy framework such as the popular AngularJS.
How to handle Selenium Page Object pattern better with Loadable Component.
The Loadable Component helps test case developers make sure that the page or a component of the page is loaded successfully. I will share my experience about the concept of the Loadable Component and Page Object patterns.
This presentation provides an overview of a Test Automation Framework with BDD and Cucumber. It also includes several open-source initiatives that Rhoynar Software Consulting (www.rhoynar.com) has been working on in the fields of QA Automation and DevOps. Lastly, it also includes links to some of the open-source projects that you can use right now for your work.
- Continuous Integration Infra a la OpenStack - https://github.com/Rhoynar/ci-infra
- An Email Verification Library in Java:
https://github.com/Rhoynar/EmailVerify
- Automatic Test Generation using Selenium WebDriver, Java and TestNG
https://github.com/Rhoynar/AutoTestR
- Barebones BDD and Cucumber Framework integrated with Java Maven and TestNG:
https://github.com/Rhoynar/qa-automation
Presentation from RTP AEM / CQ5 Meetup by Sagar Sane. This presentation provides some of the challenges and benefits of applying Test Driven Development principles to Adobe Experience Manager (AEM)/CQ5 based projects and overview of some of the tools and technologies, including Spock & Geb, which can be used for automating test cases & execution.
This session aims to shed some light on an emerging test automation tool, Cypress. Cypress resolves many of the test automation problems that a QA or a dev may face in UI Web Automation testing. And after a walkthrough, we will compare cypress with Selenium as well.
Contact us:
Website: https://www.knoldus.com/
Twitter: https://twitter.com/Knolspeak?ref_src...
Facbook: https://www.facebook.com/KnoldusSoftw...
Linkedin: https://in.linkedin.com/company/knoldus
Instagram: https://www.instagram.com/knoldus_inc...
Patterns and practices for building enterprise-scale HTML5 appsPhil Leggetter
Developing large apps is difficult. Ensuring that code is consistent, maintainable, testable and has an architecture that enables change is essential. As is ensuring that multiple developers – across multiple teams – can efficiently contribute to the same application. When it comes to large server-focused apps, solutions to some of these challenges have been tried and tested. But, how do you achieve this when building HTML5 single-page apps?
In this session, Phil will cover the experiences his team have had when building HTML5 apps consisting of more than 250k lines of JavaScript (plus HTML templates, CSS, image, config etc) that are contributed to by multiple teams across multiple companies. He will highlight signs to watch out for as your HTML5 SPA grows, and a set of patterns and practices that help you avoid problems. He will also explain the simple yet powerful application architecture that their HTML5 apps have that is core to ensuring they scale.
Finally, Phil will demonstrate how tooling can be used to support these patterns and practices, and enable a productive developer workflow where the first line of code is feature code, features can be developed and tested in isolation, code conflicts are avoided by grouping assets by feature, and features are composed into apps.
Cypress vs Selenium WebDriver: Better, Or Just Different? -- by Gil TayarApplitools
** Full webinar recording: https://youtu.be/D7vxFuwnUio **
Watch Sr. Architect Gil Tayar's special hands-on session, where he explains & demonstrates how Selenium and Cypress differ.
The session covers the following topics:
* How and why do frontend developers write tests
*Selenium architecture
*Cypress architecture
*Live Demo of Cypress - including how to write a Cypress test, and how it’s used
*How Cypress deals with flakiness
*Cypress and backdoors to development
*Visual Testing using Cypress
Appium and Selendroid
Desired Capabilities
UI Automator and Challenges faced in finding elements
Wait & Handling Keyboard
Touch Actions and Wait Actions
Android Key Events
Android and iOS Scenarios
Overall Flow using an e-Commerce app
Alternative Solution of Appium
Pros and Cons of Appium
We realise that in the world of mobile app testing, everyone is asking themselves how testing can match modern agile development processes. Establishing the right testing practices can shorten release cycles and testing time exponentially. That is what brought us to organising a session, which would be a great introduction to automated mobile testing.
Page-Object pattern is very commonly used when implementing Automation frameworks. However, as the scale of the framework grows, there is a limitation on how much reusability really happens. It inherently becomes very difficult to separate the test intent from the business domain.
I talk about this problem, and the solution I have been using - Business Layer - Page-Object pattern, which has helped me keep my code DRY.
For more details (links to slides, etc.), see my blog: http://goo.gl/biRn4n
Learn how your company can add automated testing for accessibility on all platforms. This presentation covers what Intuit has learned while working towards this goal
A method of communicating between two devices
A software function provided at a network address over the web with the service always on
It has an interface described in a machine-processable format
http://www.qualitestgroup.com/
United Global Soft
We provide QTP/QA Automation Online training by real time experts.
Contact : +91 8099902123
+1-201-710-8393
Mail Id : info@unitedglobalsoft.com
Better Page Object Handling with Loadable Component Pattern - SQA Days 20, Be...Sargis Sargsyan
One of the painful problems in Selenium automated testing is determining whether a HTML page has been loaded. This is especially the case when web application uses a JS heavy framework such as the popular AngularJS.
How to handle Selenium Page Object pattern better with Loadable Component.
The Loadable Component helps test case developers make sure that the page or a component of the page is loaded successfully. I will share my experience about the concept of the Loadable Component and Page Object patterns.
This presentation provides an overview of a Test Automation Framework with BDD and Cucumber. It also includes several open-source initiatives that Rhoynar Software Consulting (www.rhoynar.com) has been working on in the fields of QA Automation and DevOps. Lastly, it also includes links to some of the open-source projects that you can use right now for your work.
- Continuous Integration Infra a la OpenStack - https://github.com/Rhoynar/ci-infra
- An Email Verification Library in Java:
https://github.com/Rhoynar/EmailVerify
- Automatic Test Generation using Selenium WebDriver, Java and TestNG
https://github.com/Rhoynar/AutoTestR
- Barebones BDD and Cucumber Framework integrated with Java Maven and TestNG:
https://github.com/Rhoynar/qa-automation
Presentation from RTP AEM / CQ5 Meetup by Sagar Sane. This presentation provides some of the challenges and benefits of applying Test Driven Development principles to Adobe Experience Manager (AEM)/CQ5 based projects and overview of some of the tools and technologies, including Spock & Geb, which can be used for automating test cases & execution.
This session aims to shed some light on an emerging test automation tool, Cypress. Cypress resolves many of the test automation problems that a QA or a dev may face in UI Web Automation testing. And after a walkthrough, we will compare cypress with Selenium as well.
Contact us:
Website: https://www.knoldus.com/
Twitter: https://twitter.com/Knolspeak?ref_src...
Facbook: https://www.facebook.com/KnoldusSoftw...
Linkedin: https://in.linkedin.com/company/knoldus
Instagram: https://www.instagram.com/knoldus_inc...
Patterns and practices for building enterprise-scale HTML5 appsPhil Leggetter
Developing large apps is difficult. Ensuring that code is consistent, maintainable, testable and has an architecture that enables change is essential. As is ensuring that multiple developers – across multiple teams – can efficiently contribute to the same application. When it comes to large server-focused apps, solutions to some of these challenges have been tried and tested. But, how do you achieve this when building HTML5 single-page apps?
In this session, Phil will cover the experiences his team have had when building HTML5 apps consisting of more than 250k lines of JavaScript (plus HTML templates, CSS, image, config etc) that are contributed to by multiple teams across multiple companies. He will highlight signs to watch out for as your HTML5 SPA grows, and a set of patterns and practices that help you avoid problems. He will also explain the simple yet powerful application architecture that their HTML5 apps have that is core to ensuring they scale.
Finally, Phil will demonstrate how tooling can be used to support these patterns and practices, and enable a productive developer workflow where the first line of code is feature code, features can be developed and tested in isolation, code conflicts are avoided by grouping assets by feature, and features are composed into apps.
Cypress vs Selenium WebDriver: Better, Or Just Different? -- by Gil TayarApplitools
** Full webinar recording: https://youtu.be/D7vxFuwnUio **
Watch Sr. Architect Gil Tayar's special hands-on session, where he explains & demonstrates how Selenium and Cypress differ.
The session covers the following topics:
* How and why do frontend developers write tests
*Selenium architecture
*Cypress architecture
*Live Demo of Cypress - including how to write a Cypress test, and how it’s used
*How Cypress deals with flakiness
*Cypress and backdoors to development
*Visual Testing using Cypress
Appium and Selendroid
Desired Capabilities
UI Automator and Challenges faced in finding elements
Wait & Handling Keyboard
Touch Actions and Wait Actions
Android Key Events
Android and iOS Scenarios
Overall Flow using an e-Commerce app
Alternative Solution of Appium
Pros and Cons of Appium
We realise that in the world of mobile app testing, everyone is asking themselves how testing can match modern agile development processes. Establishing the right testing practices can shorten release cycles and testing time exponentially. That is what brought us to organising a session, which would be a great introduction to automated mobile testing.
Continuous integration using Jenkins and SonarPascal Larocque
Continuous Integration can help your to team release features faster. It reduces the risk of deployment issue and will speed up your development cycle. In this presentation we take a look at how Jenkins and Sonar can help you Test, Analyze, Deploy and gather performance metrics that will help your team increase their development quality and reduce deployment time
A presentation on SonarQube and my personal experience using it to learn about a project - presented at the Software Craftsmanship unconference - Socrates UK.
Design First API's with RAML and SoapUIDaniel Feist
Daniel Feist, Architect @ Mulesoft
Matti Hjelm, SoapUI Product Manager @ SmartBear
In this workshop we'll develop a real-world API together using RAML to define the interface and SoapUI to simulate and test the interface and API consumer experience before even thinking about the implementation. In doing this we'll cover the powerful design-first features of RAML as well as the tooling available to facilite this approach. We’ll also cover the mocking and prototyping features of SoapUI. Finally we'll implement, deploy and test our new API in a production environment.
GitHub pull request model and Gerrit Code Review, which one is best for you ?
What are the plus and minuses of both models ?
See how it make sense to use one or the other or even both together.
The GitHub plugin for Gerrit Code Review allows the existing developers community to start exploring code review without loosing contact with the github.com presence.
Introduction to BDD with Cucumber for JavaSeb Rose
This tutorial looks at what Cucumber is good for—and what it isn’t. It briefly covers what behavior-driven development (BDD) is and how Cucumber helps deliver on the promise of improved communication between the business and the development team. Then it dives into how Cucumber works, the specific details of the Java version (Cucumber-JVM), and what makes it different from other similar tools. Finally, it considers how Cucumber can fit into your test automation strategy and produce "living documentation" that is always up to date. You will come away with everything you need to know to be able to decide whether BDD (using Cucumber-JVM) is for you.
Describes Outside-In development and Behvaiour Driven Development. Illustrates basic Cucumber usage within a Rails app and then goes over more advanced topics such as JS as web services.
FLOW3 spearheaded a move towards Git adoption within the TYPO3 project, and we are more pleased every day with the decision to turn away from Subversion and toward GIt.
In this session I explain the workflow we adopted using Git and the code review system Gerrit. I will show how it makes collaborative development more productive and improves code quality at the same time.
Overview of Gerrit Code Review with a specific focus on its Jenkins CI integration.
See and learn how to improve your Agile application lifecycle management by making your builds more stable and your development more under control.
Gerrit Code Review allows developers to share ideas and get collective ownership of the project design and code-style.
A presentation made for the AngularJS-IL meetup group that took place in May 2014 at Google TLV Campus. its a demonstration of Unit testing an AngularJS component with jasmine and karma.
How to establish ways of working that allows shifting-left of the automation ...Max Barrass
Why Automate?
Your application will grow, you will not have enough hands
You are blocked by development
Hidden factory costs of bug-fix cycle
Why Shift-Left?
More people to negate massive inspections
Define measurable success early, work on good parts.
Reduce occurrence of defects
What is this got to do with Ways of working?
Unlock capacity
Make people smile
Is not
a Department
extra cost
a final oversight or a massive inspection
someone else’s job
Is
Everyone’s responsibility
Build into the ways of working
Everyone’s job
You want to improve your software skills. That’s a given. You may be a mentor or a manager who needs to improve the knowledge sharing among your software developers across different projects. Code Reviews can do just that while improving code quality in your projects. Code Review not only builds developer team spirit but also offers new ways to improve a software solution. You’ll walk away from this session with in-depth understanding of Code Review to strengthen your team.
Collaborating with Developers: How-to Guide for Test Engineers - By Gil Tayar Applitools
* Full webinar recording here: https://youtu.be/0NT_fmXwz1k **
"I will give a recipe that you can follow to ease your fear of the unknown: writing tests for developer code.
At the end of this session, I guarantee that you will gain a deeper understanding of different kinds of tests, know how to decipher developer terminology, and learn how to write unit, integration, browser, and E2E tests." -- Gil Tayar. Sr. Architect & Evangelist
Testing is shifting left, moving closer to testing the code itself. But while managers dictate a shift to the left, developers and testers are confused as to how exactly to test the code.
And while the backend world has established code-testing methodologies, we are still trying to figure out how to test frontend code, while ensuring effective testing procedures and processes.
This means testers need to step in and work with the frontend developers, but with an understanding of the frameworks by which frontend code is tested, the various kinds of testing that can be performed on frontend code, and which tools can be used for this.
In this hands-on session, Gil Tayar discusses various test methodologies, and how they fit together in a coherent way. Gil also includes sample code that you can use as a template in your own project -- all in order to provide you with the knowledge and tools to approach and test developer code.
Session on Testing Activities in Continuous Integration and Delivery as an Ex...Agile Testing Alliance
A session on "Testing Activities in Continous Integration and Delivery as an Exploratory Tester" by Srinivas Kadiyala, Principal Exploratory Tester at Moolya Testing. This session was a part of ATA Bangalore 25th Meetup which was held on 29th February 2020 in association with LOCUS
You want to improve your software skills. That’s a given. You may be a mentor or a manager who needs to improve the knowledge sharing among your software developers across different projects. Code Reviews can do just that while improving code quality in your projects. Code Review not only builds developer team spirit but also offers new ways to improve a software solution. You’ll walk away from this session with in-depth understanding of Code Review to strengthen your team.
Presented at the BEACON 2017 conference at Hyderabad, India on December 1 - 3, 2017; this session revisits a presentation originally delivered in 2008 with updated tools reflecting a more up-to-date Agile engineering environment.
Similar to Automated Testing With Jasmine, PhantomJS and Jenkins (20)
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...Neo4j
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfPaige Cruz
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
Removing Uninteresting Bytes in Software FuzzingAftab Hussain
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
2. 2
May 2013
Agenda
1.Who we are
2.Testing JS code and front-
end
• Testing without the browser
• Continuous Integration
2
3. 3
May 2013
1.Who we are
2.Testing JS code and front-
end
• Testing without the browser
• Continuous Integration
o Listing the types of tests
o Looking at some test tools
o Writing and running tests
Agenda
3
4. 4
May 2013
Agenda
1.Who we are
2.Testing JS code and front-
end
• Testing without the browser
• Continuous Integration
o What is a headless browser
o Scripting with a headless
browser
o Testing with a headless
browser
4
5. 5
May 2013
Agenda
1.Who we are
2.Testing JS code and front-
end
• Testing without the browser
• Continuous Integration
o Deploying code
o Integrating testing with
builds
o Review build results
5
6. 6
May 2013
Who are we?
Aisha Kaliel
drupal: akaliel
Ronn Abueg
drupal: ronnbot
twitter: @ronnbot
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration
twitter: @workatplay
facebook: www.facebook.com/workatplay
website: www.workatplay.com
6
7. 7
May 2013
We work with Drupal...
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 7
8. 8
May 2013
And other CMS/frameworks
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 8
9. 9
May 2013
And other CMS/frameworks
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 9
10. 10
May 2013
Repetitive Testing: Why not automate?
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 10
11. 11
May 2013
Why test in general
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 11
12. 12
May 2013
Automated testing...
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration
• Saves time and money
• Improves accuracy
• Increases test coverage
• Does what manual testing cannot
• Helps developers and testers
• Improves team moral
• and continuous integration is the cornerstone of Agile
development
12
13. 13
May 2013
1.Who we are
2.Testing JS code and front-
end
• Testing without the browser
• Continuous Integration
o Listing the types of tests
o Looking at some test tools
o Writing and running tests
Agenda
13
14. 14
May 2013
Types of tests
GUI
Unit
Service /
Integration
Test Pyramid
•concepted by Mike Cohn
•higher level tests at the top
•larger number of test below
•high level tests can encompass
many low level tests, e.g.
ofor form test to pass,
functional units must work
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 14
15. 15
May 2013
Tackling the testing pyramid
GUI
Unit
Service /
Integration
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 15
automating browser
JS testing
16. 16
May 2013
Testing tools
• Selenium
• Watir
• Windmill
• Mocha
• QUnit
• Jasmine
• and etc.
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 16
automating browser
JS testing
17. 17
May 2013
Testing tools
• Selenium
• Watir
• Windmill
• Mocha
• QUnit
• Jasmine
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 17
18. 18
May 2013
Comparison
Selenium
•Automates browsers
•Simulates user actions
•There's an IDE, a Firefox
extension, that allows to
record, edit, and debug tests
Jasmine
•Open-source testing
framework for JavaScript
•Does not depend on DOM or
any other JavaScript
frameworks
•Unit tests your JS code
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 18
19. 19
May 2013
Why Jasmine?
• Runs anywhere Javascript can
• Does not intrude on the
application
• Plays well with IDEs
• Encourages good testing
principles
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 19
20. 20
May 2013
What is Jasmine?
• Behavior-driven development
framework
• Does not require require a
DOM, or depend on any other
JavaScript frameworks
• Clean, obvious syntax
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 20
21. 21
May 2013
What is Jasmine?
• Behavior-driven development
framework
• Does not require require a
DOM, or depend on any other
JavaScript frameworks
• Clean, obvious syntax
• Specialized version of test-
driven development
• Focuses on the "desired
behavior" of software
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 21
22. 22
May 2013
What is Jasmine?
• Behavior-driven development
framework
• Does not require require the
DOM, or depend on any other
JavaScript frameworks
• Clean, obvious syntax
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 22
23. 23
May 2013
What is Jasmine?
• Behavior-driven development
framework
• Does not require require a
DOM, or depend on any other
JavaScript frameworks
• Clean, obvious syntax
Still has access to the DOM!
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 23
24. 24
May 2013
What is Jasmine?
• Behavior-driven development
framework
• Does not require require a
DOM, or depend on any other
JavaScript frameworks
• Clean, obvious syntax
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 24
25. 25
May 2013
Tackling the testing pyramid with Jasmine
GUI
Unit
Service /
Integration
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 25
26. 26
May 2013
Tackling the testing pyramid with Jasmine
GUI
Unit
Service /
Integration
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 26
27. 27
May 2013
Tackling the testing pyramid with Jasmine
GUI
Unit
Service /
Integration
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 27
28. 28
May 2013
Jasmine Features
• Test suites/specs
• Expectations
• Matchers
• Setup and teardown
• Spies
• Asynchronous Support
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 28
29. 29
May 2013
Feature to test: basic calculator
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 29
33. 33
May 2013
Test the calculator: Suites
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 33
34. 34
May 2013
Test the calculator: Specs
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 34
35. 35
May 2013
Testing the calculator: Expectations
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 35
36. 36
May 2013
Feature: Matchers
• expect(x).toBe(y);
• expect(x).toEqual(y);
• expect(x).toMatch(/pattern/);
• expect(x).toBeDefined();
• expect(x).toBeUndefined();
• expect(x).toBeNull();
• expect(x).toBeTruthy();
• expect(x).toBeFalsy();
• expect(x).toContain(y);
• expect(x).toBeLessThan(y);
• expect(x).toBeGreaterThan(y);
• expect(x).toBeCloseTo(y, 0);
• expect(function(){ fn(); }).toThrow(e);
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 36
37. 37
May 2013
Feature: Negative Matchers
• expect(x).not.toBe(y);
• expect(x).not.toEqual(y);
• expect(x).not.toMatch(/pattern/);
• expect(x).not.toBeDefined();
• expect(x).not.toBeUndefined();
• expect(x).not.toBeNull();
• expect(x).not.toBeTruthy();
• expect(x).not.toBeFalsy();
• expect(x).not.toContain(y);
• expect(x).not.toBeLessThan(y);
• expect(x).not.toBeGreaterThan(y);
• expect(x).not.toBeCloseTo(y, 0);
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 37
38. 38
May 2013
Testing the calculator: Matchers
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 38
39. 39
May 2013
Testing the calculator: More specs!
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 39
40. 40
May 2013
Feature: Setup and Teardown
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 40
41. 41
May 2013
Testing the calculator: Setup
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 41
42. 42
May 2013
Testing the calculator: Spies
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 42
43. 43
May 2013
Testing the calculator with DOM tests
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 43
44. 44
May 2013
Testing the calculator: Nested describe blocks
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 44
45. 45
May 2013
Testing the calculator: Setup and Teardown
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 45
46. 46
May 2013
Feature: Asynchronous Support
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 46
47. 47
May 2013
Feature: Asynchronous Support
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 47
48. 48
May 2013
Feature: Asynchronous Support
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 48
49. 49
May 2013
Testing the calculator: Reporter
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 49
50. 50
May 2013
Testing the calculator: Reporter
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 50
51. 51
May 2013
Testing the calculator: Filtering the Reporter
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 51
52. 52
May 2013
Running the test page without a browser
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 52
53. 53
May 2013
Agenda
1.Who we are
2.Testing JS code and front-
end
• Testing without the browser
• Continuous Integration
o What is a headless browser
o Scripting with a headless
browser
o Testing with a headless
browser
53
54. 54
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration May 2013
What is a headless browser
• Web browser without a GUI
• Renders web pages
• Outputs to non-humans
i.e. other software
• Useful for:
o testing
o parsing
o screen capture
54
55. 55
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration May 2013
Enter the PhantomJS
• Headless WebKit
• Scriptable with JavaScript/CoffeeScript
• Support for various web standards:
o DOM handling
o CSS selector
o JSON
o Canvas
o SVG
55
56. 56
May 2013
PhantomJS basic how to
Create a script: hello.js
console.log('Hello, DrupalCon Portland!');
phantom.exit();
Run script in the command line
phantomjs hello.js
Output
Hello, DrupalCon Portland!
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 56
57. 57
May 2013
PhantomJS modules
WebPage
A WebPage object encapsulates a web page.
System
A set of functions to access system-level functionality.
FileSystem
A set of API functions available to access files and directories.
WebServer
Using an embedded web server module called Mongoose, PhantomJS script can start
a web server.
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 57
58. 58
May 2013
PhantomJS basic how to
Page Loading: loadpage.js
var page = require('webpage').create();
page.open('http://portland2013.drupal.org', function () {
page.render('drupalcon.png');
phantom.exit();
});
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 58
59. 59
May 2013
PhantomJS basic how to
Page Loading with arguments
var page = require('webpage').create(),
system = require('system');
page.open(system.args[1], function (status) {
if (status == 'success') {
console.log('Loaded the address: ' + system.args[1]);
}
phantom.exit();
});
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 59
60. 60
May 2013
PhantomJS basic how to
Page Loading with arguments
var page = require('webpage').create(),
system = require('system');
page.open(system.args[1], function (status) {
if (status == 'success') {
console.log('Loaded the address: ' + system.args[1]);
}
phantom.exit();
});
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 60
61. 61
May 2013
PhantomJS basic how to
Page Loading with arguments
var page = require('webpage').create(),
system = require('system');
page.open(system.args[1], function (status) {
if (status == 'success') {
console.log('Loaded the address: ' + system.args[1]);
}
phantom.exit();
});
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 61
62. 62
May 2013
PhantomJS basic how to
Page Loading with arguments
var page = require('webpage').create(),
system = require('system');
page.open(system.args[1], function (status) {
if (status == 'success') {
console.log('Loaded the address: ' + system.args[1]);
}
phantom.exit();
});
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 62
63. 63
May 2013
PhantomJS basic how to
Run script in the command line
phantomjs loadpage.js http://www.google.com
Output
Loaded the address: http://www.google.com
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 63
64. 64
May 2013
PhantomJS basic how to
Code Evaluation
var page = require('webpage').create();
page.open(url, function (status) {
var title = page.evaluate(function () {
return document.title;
}); console.log('Page title is ' + title); });
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 64
65. 65
May 2013
Running Jasmine with PhantomJS
Github: phantomjs / examples / run-jasmine.js
Located at: https://github.com/ariya/phantomjs/blob/master/examples/run-jasmine.js
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 65
66. 66
May 2013
Running Jasmine with PhantomJS
From https://github.com/ariya/phantomjs/blob/master/examples/run-jasmine.js
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 66
67. 67
May 2013
Running Jasmine with PhantomJS
From https://github.com/ariya/phantomjs/blob/master/examples/run-jasmine.js
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 67
68. 68
May 2013
Running Jasmine with PhantomJS
From https://github.com/ariya/phantomjs/blob/master/examples/run-jasmine.js
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 68
69. 69
May 2013
Running Jasmine with PhantomJS
From https://github.com/ariya/phantomjs/blob/master/examples/run-jasmine.js
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 69
70. 70
May 2013
Running Jasmine with PhantomJS
From https://github.com/ariya/phantomjs/blob/master/examples/run-jasmine.js
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 70
71. 71
May 2013
Running Jasmine with PhantomJS
From https://github.com/ariya/phantomjs/blob/master/examples/run-jasmine.js
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 71
72. 72
May 2013
Running Jasmine with PhantomJS
From https://github.com/ariya/phantomjs/blob/master/examples/run-jasmine.js
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 72
73. 73
May 2013
Running Jasmine with PhantomJS
From https://github.com/ariya/phantomjs/blob/master/examples/run-jasmine.js
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 73
74. 74
May 2013
Running Jasmine with PhantomJS
From https://github.com/ariya/phantomjs/blob/master/examples/run-jasmine.js
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 74
75. 75
May 2013
Running Jasmine with PhantomJS
From https://github.com/ariya/phantomjs/blob/master/examples/run-jasmine.js
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 75
76. 76
May 2013
Running Jasmine with PhantomJS
From https://github.com/ariya/phantomjs/blob/master/examples/run-jasmine.js
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 76
77. 77
May 2013
Running Jasmine with PhantomJS
From https://github.com/ariya/phantomjs/blob/master/examples/run-jasmine.js
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 77
78. 78
May 2013
Running Jasmine with PhantomJS
From https://github.com/ariya/phantomjs/blob/master/examples/run-jasmine.js
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 78
79. 79
May 2013
Running Jasmine with PhantomJS
From https://github.com/ariya/phantomjs/blob/master/examples/run-jasmine.js
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 79
80. 80
May 2013
Running Jasmine with PhantomJS
From https://github.com/ariya/phantomjs/blob/master/examples/run-jasmine.js
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 80
81. 81
May 2013
Running Jasmine with PhantomJS
From https://github.com/ariya/phantomjs/blob/master/examples/run-jasmine.js
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration
...
...
...
81
82. 82
May 2013
PhantomJS features
Headless web testing
Lightning-fast testing without the browser with various test frameworks.
Page automation
Access and manipulate web pages with the standard DOM API, or with usual
JavaScript libraries.
Screen capture
Programmatically capture web contents including CSS, SVG and Canvas.
Network monitoring
Automate performance analysis, track page loading and export as standard HAR
format.
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 82
83. 83
May 2013
PhantomJS custom modules (v1.7)
Require
Users can reference their own modules from the file system using require function.
General usage
var server = require('webserver').create();
var Awesome = require('MyAwesomeModule');
Awesome.do();
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 83
84. 84
CI
May 2013
Lets integrate it with Jenkins
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 84
85. 85
May 2013
Agenda
1.Who we are
2.Testing JS code and front-
end
• Testing without the browser
• Continuous Integration
o Deploying code
o Integrating testing with
builds
o Review build results
85
86. 86
May 2013
Meet Jenkins
• Leading open-source
Continuous Integration server
• Built with Java
• Plug-in support
• RSS/E-mail/IM support
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 86
87. 87
May 2013
Jenkins in action - dashboard
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 87
88. 88
May 2013
Jenkins in action - project
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 88
89. 89
May 2013
Jenkins in action - build
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 89
90. 90
May 2013
Jenkins in action - build detail
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 90
91. 91
May 2013
Deploying code - console output of build
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 91
92. 92
May 2013
PhantomJs & Jasmine with XML
Github: detro / phantomjs-jasminexml-example
Located at: https://github.com/detro/phantomjs-jasminexml-example
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 92
93. 93
May 2013
Jasmine Reporter Plugin
Github: ../ test / lib / jasmine-reporters
Located at: https://github.com/detro/phantomjs-jasminexml-example/tree/master/test/lib/jasmine-reporters
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 93
94. 94
May 2013
Jasmine Reporter Plugin
Github: ../ test / lib / jasmine-reporters
Located at: https://github.com/detro/phantomjs-jasminexml-example/tree/master/test/lib/jasmine-reporters
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 94
95. 95
May 2013
PhantomJS XML Runner
Github: ../ test / phantomjs_jasminexml_runner.js
https://github.com/detro/phantomjs-jasminexml-example/blob/master/test/phantomjs_jasminexml_runner.js
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 95
96. 96
May 2013
PhantomJS XML Runner
Github: ../ test / phantomjs_jasminexml_runner.js
https://github.com/detro/phantomjs-jasminexml-example/blob/master/test/phantomjs_jasminexml_runner.js
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 96
97. 97
May 2013
PhantomJS XML Runner
Github: ../ test / phantomjs_jasminexml_runner.js
https://github.com/detro/phantomjs-jasminexml-example/blob/master/test/phantomjs_jasminexml_runner.js
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 97
98. 98
May 2013
PhantomJS XML Runner
Github: ../ test / phantomjs_jasminexml_runner.js
https://github.com/detro/phantomjs-jasminexml-example/blob/master/test/phantomjs_jasminexml_runner.js
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 98
99. 99
May 2013
PhantomJS XML Runner
Github: ../ test / phantomjs_jasminexml_runner.js
https://github.com/detro/phantomjs-jasminexml-example/blob/master/test/phantomjs_jasminexml_runner.js
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 99
100. 100
May 2013
PhantomJS XML Runner
Github: ../ test / phantomjs_jasminexml_runner.js
https://github.com/detro/phantomjs-jasminexml-example/blob/master/test/phantomjs_jasminexml_runner.js
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 100
101. 101
May 2013
PhantomJS XML Runner
Github: ../ test / phantomjs_jasminexml_runner.js
https://github.com/detro/phantomjs-jasminexml-example/blob/master/test/phantomjs_jasminexml_runner.js
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 101
102. 102
May 2013
PhantomJS XML Runner
Github: ../ test / phantomjs_jasminexml_runner.js
https://github.com/detro/phantomjs-jasminexml-example/blob/master/test/phantomjs_jasminexml_runner.js
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 102
103. 103
May 2013
Running PhantomJS in Jenkins: configuration
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 103
104. 104
May 2013
Running PhantomJS in Jenkins: Console Output
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 104
105. 105
May 2013
Publishing test results in Jenkins: configuration
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 105
106. 106
May 2013
Publishing test results in Jenkins: configuration
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 106
107. 107
May 2013
Publishing test results in Jenkins: Console Output
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 107
108. 108
May 2013
Review build and test results
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 108
109. 109
May 2013
Review build and test results
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 109
110. 110
May 2013
Review test results
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 110
111. 111
May 2013
Review test results
Who we are | Testing JS Code & Front-End | Testing without the Browser | Continuous Integration 111
112. 112
May 2013
Agenda
1.Who we are
2.Testing JS code and front-
end
• Testing without the browser
• Continuous Integration
112
113. 113
May 2013
Agenda
1.Who we are
2.Testing JS code and front-
end
• Testing without the browser
• Continuous Integration
o Wrote and ran tests in
Jasmine
o We are able to do various
types of testsautomatically
o This is more accurate than
manual testing
113
114. 114
May 2013
Agenda
1.Who we are
2.Testing JS code and front-
end
• Testing without the browser
• Continuous Integration
o Executed Jasmine tests via
PhantomJS
o The tests can be ran by
non-humans
o This saves us resources
114
115. 115
May 2013
Agenda
1.Who we are
2.Testing JS code and front-
end
• Testing without the browser
• Continuous Integration
o Added tests execution
during builds on Jenkins
o Reviewed test results in
Jenkins after builds
o Allows us to be more agile
115
117. 117
May 2013
Thank you!
Evaluate this session at:
portland2013.drupal.org/schedule
Enjoy the rest of
Drupalcon 2013 Portland
117
Editor's Notes
Who we are Testing JS code and front-end Listing the types of tests Looking at some test tools Writing and running tests Testing without the browser What is a headless browser Scripting with a headless browser Testing with a headless browser Continuous Integration Deploying code Integrating testing with builds Review build results
Who we are Testing JS code and front-end Listing the types of tests Looking at some test tools Writing and running tests Testing without the browser What is a headless browser Scripting with a headless browser Testing with a headless browser Continuous Integration Deploying code Integrating testing with builds Review build results
Who we are Testing JS code and front-end Listing the types of tests Looking at some test tools Writing and running tests Testing without the browser What is a headless browser Scripting with a headless browser Testing with a headless browser Continuous Integration Deploying code Integrating testing with builds Review build results
Who we are Testing JS code and front-end Listing the types of tests Looking at some test tools Writing and running tests Testing without the browser What is a headless browser Scripting with a headless browser Testing with a headless browser Continuous Integration Deploying code Integrating testing with builds Review build results
We're both developers from Work at Play. Work at Play is digital agency that provides clients with custom engagement solutions that decrease churn and increase long term customer value. This basically translates to either building apps and using social marketing, or both.
Just some of the Drupal sites we've worked on ABCSpark.ca BarbieCollector.com, Mattel.com, MonsterHigh.com, HotWheels.com (no image), Knowledge.ca NeedForSpeed.com
Some other CMS and frameworks: Yii, CodeIgniter Java Spring, Alfresco Django PhoneGap Not even including some other ones, like Kohana and TYPO3 Then there are the JS frameworks: BackboneJS KendoUI ExtJS jQuery (UI, Mobile)
Now, regardless of the backend, framework the result is lots and lots of HTML and JS (and CSS).
[pause for about 20 or so seconds] / "I'll let you read this comic first" You could test your pages via manual click tests, but that's ridiculous. You have to do these manual tests every time your code evolves with new features, or improvements, to ensure that your code is still working as it should. Which can become frustrating with the amount of time and effort an individual person has to put in in order to do so. So why not automate it and remove that pain? Hence, this session on AUTOMATED TESTING WITH JASMINE AND PHANTOMJS.
This is our company slogan. It is painted on the wall We work smart partly by automating tests... This gives us the opportunity to play hard with new and interesting technology
Saves time and money test to ensure quality but Manually repeating these tests is costly and time consuming. automated tests can be run over and over again at no additional cost and they are much faster than manual tests Improves accuracy tester will make mistakes during monotonous manual testing Automated tests perform the same steps precisely every time they are executed Increases test coverage automated testing can increase the depth and scope of tests to help improve software quality Lengthy tests that are often avoided during manual testing can be run unatten Does what manual testing cannot simulation load and stress tests Helps developers and testers developers can share test codes; tests can be ran by developers and not just testers Improves team moral with the the time saved, the team can work on more building and solving interesting problems instead of manually testing over and over again and continuous integration is the cornerstone of Agile development Agile practices rely on fast feedback, achieved by testing each incremental change Automation allows us to break free of waterfall's "coding phase / testing phase" paradigm, turning them into an integrated development cycle
What we have here is a lit of types of tests. Of course, there are other types of tests, such as stress tests, but they're out of scope. As you can see, unit tests are at the bottom of the list because they're lower level tests compared to GUI tests. The test pyramid is a concept developed by Mike Cohn , described in his book Succeedin g with Agile . It's a pyramid because you have much more low level tests at the base. And as you go up, there are less and less tests of that type. Also, the higher level tests can encompass one to many lower level tests.
On the front-end or client side, the way these tests are normally done are: GUI - automating the browser by scripting it or recording actions Unit test - by writing and running tests in JS
Now, here are some testing tools; there are many more obviously. In this list, the ones at the top like selenium and watir does testing by automating the browser. The ones below are mainly for unit testing your JS/HTML code. Some tools, like Windmill and Mocha, can do a both.
In this session, we're obviously be looking at Jasmine, but we'll also briefly look at Selenium, which sits on the other side of the spectrum.
Compare with selenium. More info on selenium. Highlight what selenium does better than jasmine and vice-versa.
Many other javascript testing frameworks only work from within a browser. Some don't support testing asynchronous code like event callbacks and some still have syntax that's hard for JS developers or IDEs to understand. Jasmine aims to run on any JavaScript-enabled platform, to not intrude on the application nor the IDE , and to have easy-to-read syntax. Jasmine was designed with principles in mind for what makes a good up a JS testing framework. With the goals for writing jasmine, [pause] one being that it should encourage good testing practices. [next slide]
More about what Jasmine is, and what are some of those principles and goals... Well first, Jasmine is a behaviour driven development framework for testing javascript code. [next slide]
- A Behaviour driven development framework is a specialized and continued version essentially of TDD - as it makes more specific choices than TDD does - it focuses on the desired behaviours of software, borrowing from agile software development - so tests are specified in terms of those "desired behaviors", using sentences starting with the word "should"
It is designed to not be tied to any browser, framework, platform, or host language. You don't need any other javascript frameworks or libraries in order to start working with jasmine. It only needs them if it is testing it. It also doesn't need the DOM, which means that we can do not need to rely on it to write unit tests and can just interact with only the javascript objects that we need and want to test. But though it doesn't require it.... [next slide]
... it can still access the DOM if needed! It can work anywhere JavaScript can run, including browsers, servers, phones, etc. We still have the ability to test the DOM too, and can use any libraries or frameworks we choose to.
Jasmine is designed with the principle that a good JavaScript testing framework should also have clean, idiomatic and unsurprising syntax in order for it to be easy to read and that writing jasmine should be simple to get started with.
Looking back the the test pyramid, we know Jasmine can do unit tests. Not requiring the DOM means we can test our JS objects.
With Jasmine's async support, we can test backend services and APIs by making AJAX calls.
Having access to the DOM means being able to test the UI, e.g. check if certain Drupal blocks exist on a page, fill in a form and submit it, and so on.
So, lets dive into jasmine and how to write these tests: Jasmine has the usual set of features (for a js testing framework), such as the following, which we will go over in more detail later. It has: - test suites and specs - the "expect" test method - various matchers which are included but also the ability to create your own custom matchers - setup and teardown functions - spies - asynchronous support - Plus many more... For this session, we'll look at using these to test a basic calculator that we set up in Drupal.
Here you see the sample Drupal site and calculator.
On closer look, the calculator provides 4 mathematical operations and two input variables, that output the results of the chosen operation to the final field.
This JavaScript object implements our simple calculator.
Here we show how we utilize the calculator object. We have one or more form with the class 'calc' and we simply call 'set' and 'calculate' functions of the calculator object when certain events are triggered. What we'll do initially is to unit test the calculator object (shown in the previous slide).
[after setting up the HTML runner file that actually runs the Jasmine tests, which you can take a look at in the jasmine documentation or sample files] (First thing, ) We can create the test suite to hold all our tests, or "specs", for the calculator. It begins with a call to jasmine's "describe" function. You pass it two parameters, a string and a function. The first parameter, the string, is the name or title of the suite. The function is container for the block of code that implements the tests of the suite. This is where you would start to put in your specs.
A spec is a function very similar to a suite, it also has two parameters with a string and a function, Though it is defined by calling the jasmine's function "it". The string is the title of the spec and what exactly it is testing. As we mentioned before, since it is a behaviour driven testing framework, it is likely a statement that starts with "should" In this case "should be able to add two numbers together". The function is where the block of code to execute that spec is placed, and we begin to interact with our calculate object.
Expectations are written with the function expect. It takes a value, the actual, which we will implement matchers on to evaluate it.
Matchers implement a boolean comparison between that "actual" value and the expected value we compare to. Jasmine has a set of matchers included that we can begin to use, which we can see here. toBe: compares x and y and passes if they are the same toEqual: compares x and y and passes if they are equivalent toMatch: compares x to the string or regular expression and passes if they match toBeDefined: passes if x is not undefined toBeUndefined: passes if x is undefined toBeNull: passes if x is null toBeTruthy: passes if x evaluates to true toBeFalsy: passes if x evaluates to false toContain: passes if the array or string x contains y toBeLessThan: passes if x is less than y toBeGreaterThan: passes if x is greater than y toThrow: passes if the function fn throws an exception e when executed
But any matcher can evaluate to a negative assertion as well by chaining the call to expect with a not before calling that matcher. It compares the "actual" value to the expected and then passes if they are NOT equivalent (in this case).
In our calculator example, we use the toEqual matcher after we using the calculator object to calculate using two numbers (10 and 15) which were in turn ADDED together The result of that action we use as the "actual" value in the expect function then check that the expected value EQUALS in this case 25.
We can start adding more specs for other operations of the calculator within our test suite. Such as ones for subtracting and multiplying seen here. They both use the expect and toEqual matcher to see if the actual value returned from our calculator object is the value we expected. But alternatively, we could possibly use other matchers such as toBeLessThan for the subtraction spec, to test that the "actual" value is less than x. (TODO: Should this be another slide?)
The next feature is setup and teardown. These are functions are called before and after each test within a test suite. Here you see the basic syntax for the setup, which is function beforeEach. Then the teardown, which is function afterEach.
We can use the beforeEach to help set up variables or the state of the code we want before each of our tests. So now we don't have to repeat that code in every spec we write. For these tests, we use the setup to create a new instance of our 'calculator' object.
Another functionality to use is Spies. Spies act as test doubles and can sub any function and track calls to it with all the arguments. Spies have their own set of special matchers to be used to interact with them. Here we set a spy to make sure that the function 'add' is called when the function 'calculate' is executed with the "+" symbol. It should be noted that we can no longer do expects on the value returned for z, as the function "add" was subbed out with the spy. Which means that when add is called by calculate it doesn't call the calculators add function which returns the calculated value. So we no longer check if the value of z is correct in this particular tests. Hence why the spy is in a separate test than those expectation tests.
Now, we can move onto testing the calculator UI. This time we will interact with the DOM using jQuery for our tests. (We will need to remember to include jQuery in our runner HTML file to do this.) In this test: we get the form populate fields with values submit the form by triggering a change then check the resulting value of the answer field When the test pass, we know a few things: our form and input fields do exist the event handlings are good and the calculator object still works while implemented with the DOM
We can add the DOM tests to the rest of our calculator test suite. To do this, we can nest describes within our main test suite: we can group these DOM specific tests together we can add it's own set of beforeEach/afterEach functions for this set of tests
Here we use the beforeEach to set the form variable that we use it our DOM tests. But we also start using the afterEach as well. We use it to reset the form and it's input fields after we finish each test. So the next test we write starts fresh and doesn't have values that could mess up its results.
Jasmine also has support for running specs that require asynchronous operations. Specs can be written with a set of blocks with calls to the function "runs" and "waitsFor". For our calculator example, it does not have any operations that are asynchronous, so here is another example using a timeout to demonstrate this functionality. The first runs holds the asynchronous call, in this case the timeout function.
The waitsFor block has the latch function, which polls until it returns true. Which we set with the timeout in the runs section above. The waitsFor block also takes an optional failure message and a timeout. If the timeout for the waitsFor expires, the spec fails with the error message.
But once the asynchronous conditions are met based on the return in the latch function of waitsFor, another "runs" block defined the final tests. Which is usually the expectations based on the state after the original async call returns. For example, asynchronous operations is very useful when testing things like: - code that requires loading, for instance: downloads, video or music players, and many others - or code with animations that you may have to wait for it to finish before testing the state after that animation - or when submitting forms - or even testing ajax or other api calls (via javascript)
The javascript file with the suite, is then included via a <script> tag within the HTML runner file. The runner file is used to get Jasmine to execute the suite or suites and all its specs, then use an instance of a reporter such as the HTMLReporter, which is included with jasmine, that calls to get the results of each spec and each suite and finally present those results to the user. The HTMLReporter displays all the suites with each individual sub-suite and spec that ran. It displays in green when the test passes, or red if it fails. Plus it gives a final count of the number of specs that ran, and how many were successful. In this case, we ran all the calculator specs within the suites, totally 6, and all passed!
The javascript file with the suite, is then included via a <script> tag within the HTML runner file. The runner file is used to get Jasmine to execute the suite or suites and all its specs, then use an instance of a reporter such as the HTMLReporter, which is included with jasmine, that calls to get the results of each spec and each suite and finally present those results to the user. The HTMLReporter displays all the suites with each individual sub-suite and spec that ran. It displays in green when the test passes, or red if it fails. Plus it gives a final count of the number of specs that ran, and how many were successful. In this case, we ran all the calculator specs within the suites, totally 6, and all passed!
We can also delegate filtering of the specs to the reporter. This allows for us to click on individual suites or specs and run to see that particular subset, and not the entire list of all tests. We can choose to rerun just the "should be able to add two numbers together" test when that is the only one we care about at the moment.
TODO: Image: Possible idea is to have the code for the test suites then a funnel to go into the headless browser? Now we have a suite of tests for our calculator code that we can run in the browser, we can check if our code works and continues to run as expected. But this still is partly a manual process. The test pages are in HTML and need a browser to run it, with a person still that is to initiate the process and run that page manually. To further automate this process, we want to start to be able to have the tests run but outside of a browser now, something that a non-human could execute. Such as in the command line with the use of a headless browser.
[Useful for]..... particularly for AJAX heavy web sites/apps
So, there is PhantomJS! It is a Headless browser, specially a headless webkit. Which is essentially LIKE Chrome, But - no window - no browser, - and an invisible viewport [Accordingly] It has a javascript API, and so you can feed scripts written with javascript or even coffeescript into the command-line based utility. It has support for various web standards such as: DOM handling, CSS selector, JSON, Canvas, SVG... Really, anything that webkit can do!
So, let's try out a basic example of how to execute JS based programs with PhantomJS. We start off by creating a script file, in this example called hello.js With javascript we can output a message from it using console.log. Then tell phantom to stop and exit To run the script that was just created, we call phantomjs in the command line and pass the path to the script file we just created. It runs our script, and as instructed outputs the message we specified.
PhantomJS uses Module API, which is modeled after CommonJS Modules that are available. This set here is available up through PhantomJS 1.6. ... [read slides]... [Webpage] (read above) is an object that encapsulates a web page (and its functionalities) [System] holds the set of function to access the system level functionality... which includes the os, platform, and env, along with access to the arguments passed into the script [FileSystem] (read above) with the ability to read and write [WebServer] on the other hand is experimental. It uses an embedded web server module called Mongoose, which phantomJS can script to start a web server with. This is intended for ease of communication between PhantomJS scripts and the outside world but is not recommended for use as a general production server.
So using those modules we can use phantomJS for things like: Page loading This example shows a script called loadpage.js In the script, we load and create one of phantomJS's webpage objects. We use the object to load the URL to drupalcon portlands website, using page.open for the first parameter. In the second parameter is the function which is called after the page is loaded, or finishes attempting to load. We explain this a bit further in the next slide. After the page loads though in this case, we render the website to a png image called drupalcon.png Then finally exit from phantomjs
But to further extend the page loading script, we want to pass in arguments and tell our script dynamically which url we want it to load in a page. In order to do so, we take the basics of the last example and modify it.
We start by also creating one of PhantomJS's system objects after the page.
Then we get the first argument, AFTER the path to the script that we are running, from the system.args. The arguments include the path to the script as the first argument passed to phantomJs. We use the URL argument passed to be loaded into the WebPage object
As mentioned before, the second parameter is a function that is called after a page is loaded, but is really called after the attempt to load has happened. So it will also be called if the page failed to load. It passes 1 argument to that function for the status of the page load. We can use it to see if the page load was successful or not, then do different actions depending on it. Here, we output to the console that the address was loaded if it was successful.
Now we can run this example, much like we did before. We call phantomjs and pass in the script path as the first argument, but then as the second argument we pass in the URL that we are going to load. For example, we pass in the URL for google. PhantomJS runs our script, using the google.com parameter to load that page and when it successfully loads, outputs to the console window the message that the address was loaded.
Using PhantomJs's webpage object, we can also conduct code evaluation on the page that was loaded. After a page was successfully loaded, we can use phantom to access and manipulate the code on that page within the page.evaluate function. From there we can return elements or information from the page loaded, for instance the document title, and set it to a variable within our phantomjs script. Then output that title to the console when we run it.
Page loading and code evaluation are used in conjunction to run the jasmine tests that we created, then parse the results from the page so that we can output them in a way that is useful for automation. Thankfully, phantomJS has a list of example scripts, one of which is to run tests with jasmine, which you can find on github under the examples folder. So we can take a look now at the code for run-jasmine.js
This example, loads the page to the jasmine runner file that runs the tests which we pass in as the first argument after this script path. Looking at the code we see the what we learned before with page loading....
the webpage object being created....
then opening that webpage with the variable to the jasmine tests. After page loads successfully, the script needs to wait for the tests to finish executing.....
It does this with its own "waitsFor" function which you pass the check for the condition of what we are waiting for as the first parameter...
...it uses code evaluation with page.evaluate to do this....
...and test when the pending class on the body is removed. Which is done by jasmine's HTML Reporter after all the tests done running.
Here we can take a bit closer look at this WaitsFor function. [next]
Not only does it pass the waiting condition, but you also pass it a "onready" function that is called when the waiting condition is met...
and an optional time in milliseconds for how long the script should wait for that condition to be met. By default it is set to 3 seconds.
When that time is reached, and the waiting condition is still not returned, it tells the script to stop, and exit with a failure.
Otherwise, it calls our onReady function that we passed in.
Here we see that onready function to be called.
it uses page.evaluate again to access the page the DOM of the finished runner HTML.
It parses the HTML to identify the number of successes
and failures. Which are reported to the console.
Depending on if there were any failures, it exits phantom with the exit status of 0 or 1. 1 being if any errors existed.
So in conclusion, lets take a quick look at some of the features that we went over for PhantomJS and what it is capable of. You can do a lot more with phantomJS outside of using it to run javascript tests with jasmine. phantomJS can do fast headless testing with other test frameworks too such as Mocha, Qunit, WebDriver, etc But using what we have shown today, it can be used for many different possibilities. You don't have to stop at automating tests! We discussed how it handle Page Automation by accessing and manipulating web pages with the DOM API. But it can also do this with the usual javascript libraries like jQuery. It can also be used for Screen capture, to capture web contents including CSS, SVG, and Canvas For example, there is already a Drupal module out there using PhantomJS, called PhantomJS Capture It has provides integration with PhantomJS and the screen capture abilities to allow you to take screenshots of homepages and use them as images on your site. Finally, it is also possible to use it for Network monitoring. Such as automating performance analysis, or track page loading. You can also use YSlow to integrate with PhantomJS (and even Jenkins) which would be useful for automating web performance.
Plus With version 1.7, phantomJS also adds the possibility of creating and using custom modules. Users can reference and load their own modules using the require function in phantomJS scripts, just as they would with the included Phantom ones Seen here we use the require function to load the webserver, but also our "MyAwesomeModule" in the same way then call functions from the loaded object [pause] But let's head back to finish automating our tests...
So to summarize, we can now see how to execute the Jasmine tests, and run them in the command line with PhantomJS. We can easily run our tests as part of a script! Since we use Jenkins for deployments, we can run the tests after commits as part of the [Next slide] Continuous Integration.
Leading open-source Continuous Integration server Built with Java and it's easy to install and configure Plug-in support with over 400 plug-ins RSS/E-mail/IM support for notifications
Here's the dashboard of jenkins Currently mousing over the project and we see a dropdown menu.
Project view. Build history on the bottom left.
We clicked "Build Now" We see a progress bar. We can also see a more detail view of the progress by clicking the build in the build history
Here's the details of the build as it happens. We can click on the console output to see what the server is doing
Here's the console output We can see that it did a git pull
Right now, the script we created in teh previous section, we had the results from the jasmine tests get spit out with PhantomJS into the console window. For Jenkins to properly understand those results we need to convert the output into something that jenkins can use. The way to so that is to structure the results of the tests into XML. There is a couple version already on github that people have contributed, but the one we have used it from the phantomJs - Jasmine XML Example Project. It is an example project layout that is destined to be used standalone or with Maven, but also works with Jenkins! From this repo we essentially use 2 scripts, a new reporter, and an altered phantomjs script.
First, is a plugin for jasmine for a new reporter. It saves the results of the tests which the phantomjs script will interact with.
To enable it, first include the script in the header, but then before executing jasmine, replace the HTMLReporter, with the Trivial Reporter from Jasmine and then this new PhantomJSReporter.
Next, we take a look at the phantom JS script at phantomjs_jasminexml_runner.js We could take this script as is which is really close but uses different waiting functions, or we can the take parts of it we need and replace sections in our current script.
First, we change the code in the evaluate function for the waitsFor to instead, check the variable in jasmine was set from the new phantom js reporter we use instead of checking the DOM.
When it finishes, the waitsfor function needs to call new code for our on ready function
we replace it so that we can get the suite results from the new phantom js reporter instead of parsing the HTML of the page.
We loop through these results, which returns a list with items that contain an: xml file name Which comes from the name of the suite that the results are for. For our script we will only have one, the one called calculator.
The result also include the xml body. It contains the formatted xml with all the specs for that suite: For each spec, it has the title, the passing status, and any possible error messages.
Our script can now takes that xml file name and create a new file using the PhantomJS file system. Then save content with the given xml body to the created file.
Finally, the script to exit changes a bit. It will get the passed status from the new reporter which will be used to return 0 or 1 again. so to summarize, with the changes that were made now when the script runs: it will run the jasmine tests, get the results of those tests with the new reporter and NOT the DOM, it will structure those results into xml and save the xml into files that jenkins can later use to parse to display the results.
Now we can setup Jenkins to use our new script during the build phase. On the confirguration page of Jenkins, we can select to execute a shell command after we build and push our code. Here we tell phantomjs to run our script, passing in the path to our script, referencing it from the jenkin project workspace. Then we will aslo pass in the URL to our jasmine test runner page, and finally a folder within the workspace for the script to save all the xml result files to.
Now when we build, and see the progress in the console output page, we can watch as the jasmine tests run which still show messages in the console for what tests are currently running, and whether or not they passed.
But for Jenkins to start using the results to display outside of the console ouput, we need to tell it to parse the XML files. We go back to the configuration page, and from within the post build actions section, we can add "Publish xUnit test result report" action, and select to use a JUnit Pattern. The resulting input field is where we enter that folder to all our xml files that we passed to our phantom script for the shell command configuration. We can leave the rest of the checkboxes as the defaults.
But now we can also choose to configure how many jasmine failures it should take to mark the build as having failed. In most cases we can set the threshold to 1 for all fields. So that if one or more failures happen during a build, that build is marked as broken. We can also use this to trigger events such as notifications if we needed.
But for now, we can see Jenkins parsing our results after the tests run in the console window.
Then when it is finished we can see the Test Result section on the main build page, with the count of failures in brackets beside it. (Here we see that there we no failures that happened for our build)
Then when it is finished we can see the Test Result section on the main build page, with the count of failures in brackets beside it. (Here we see that there we no failures that happened for our build)
If we click on the Test Result link, we can see more information about the tests that ran. The bar at the top shows the precentage of successful tests to failed tests. Blue for the percentage of successful ones, and red for the failed. If there were any failures, for each of them they would show in the table beneith All Tests. You could click on any of the failed tests, and see a page that displayed the error message from jasmine for why it failed. But in this case, all our tests were successful so we don't see anything here.
If we click on the Test Result link, we can see more information about the tests that ran. The bar at the top shows the precentage of successful tests to failed tests. Blue for the percentage of successful ones, and red for the failed. If there were any failures, for each of them they would show in the table beneith All Tests. You could click on any of the failed tests, and see a page that displayed the error message from jasmine for why it failed. But in this case, all our tests were successful so we don't see anything here.