What does it take to win the Kaggle/Yandex competition
•Download as PPTX, PDF•
6 likes•4,614 views
This document summarizes the key aspects of winning the Kaggle/Yandex competition to re-rank search results according to personal user preferences. It describes the goal of predicting user pertinence for URLs to improve search rankings. It then outlines the team's approach, which involved constructing many features, using Dataiku Science Studio for modeling, and optimizing models like Random Forest and LambdaMART (which won) to directly improve the NDCG ranking metric. The team worked collaboratively over 9 months to achieve the top ranking.
Report
Share
Report
Share
Recommended
Ben Hamner, Co-founder and CTO, Kaggle at MLconf SF - 11/13/15
Lessons learned from Running Hundreds of Kaggle Competitions: At Kaggle, we've run hundreds of machine learning competitions and seen over 80,000 data scientists make submissions. One thing is clear: winning competitions isn't random. We've learned that certain tools and methodologies work consistently well on different types of problems. Many participants make common mistakes (such as overfitting) that should be actively avoided. Similarly, competition hosts have their own set of pitfalls (such as data leakage).
In this talk, I'll share what goes into a winning competition toolkit along with some war stories on what to avoid. Additionally, I’ll share what we’re seeing on the collaborative side of competitions. Our community is showing an increasing amount of collaboration in developing machine learning models and analytic solutions. I'll showcase examples of this and discuss how these types of collaboration will improve how data science is learned and applied.
Kaggle presentation
Winning Kaggle competitions involves getting a good score as fast as possible using versatile machine learning libraries and models like Scikit-learn, XGBoost, and Keras. It also involves model ensembling techniques like voting, averaging, bagging and boosting to improve scores. The document provides tips for approaches like feature engineering, algorithm selection, and stacked generalization/stacking to develop strong ensemble models for competitions.
Kaggle Winning Solution Xgboost algorithm -- Let us learn from its author
This document provides an overview of XGBoost, an open-source gradient boosting framework. It begins with introductions to machine learning algorithms and XGBoost specifically. The document then walks through using XGBoost with R, including loading data, running models, cross-validation, and prediction. It discusses XGBoost's use in winning the Higgs Boson machine learning competition and provides code to replicate its solution. Finally, it briefly covers XGBoost's model specification and training objectives.
Tips for data science competitions
This document provides tips for winning data science competitions by summarizing a presentation about strategies and techniques. It discusses the structure of competitions, sources of competitive advantage like feature engineering and the right tools, and validation approaches. It also summarizes three case studies where the speaker applied these lessons, including encoding categorical variables and building diverse blended models. The key lessons are to focus on proper validation, leverage domain knowledge through features, and apply what is learned to real-world problems.
Dataiku at SF DataMining Meetup - Kaggle Yandex Challenge
This is a presentation made on the 13th August 2014 at the SF Data Mining Meetup at Trulia. It's about Dataiku and the Kaggle Personalized Web Search Ranking challenge sponsored by Yandex
Personalized Re-Ranking of Documents
This document discusses personalized search and re-ranking search results based on a user's profile and past behavior. It describes extracting features from query logs covering 27 days of search data to train a classifier. Features include documents clicked and time spent by both the same and different users for a given query. The model is trained using LambdaMART ranking algorithm on 24 days of data and validated on 3 days. It then re-ranks the top 10 search results for test queries based on the extracted features to provide a personalized search ranking. Evaluation on a test platform showed an NDCG score higher than the baseline, indicating more relevant results.
Quality Jam 2017: Jesse Reed & Kyle McMeekin "Test Case Management & Explorat...
Jesse Reed, QA Director at Questar, and Kyle McMeekin discuss how Questar made the switch to qTest and the key factors you should consider in test case management and exploratory testing.
Florian Douetteau @ Dataiku
The document describes a Kaggle competition to personalize web search results using historical search engine logs. A team called Dataiku used supervised learning techniques like random forests and LambdaMART gradient boosted trees to re-rank web search results for new queries based on features from past user search behavior, achieving a 1st place result. Their approach involved prototyping with fast methods, optimizing models with boosting, and being systematic about feature selection and validation.
Recommended
Ben Hamner, Co-founder and CTO, Kaggle at MLconf SF - 11/13/15
Lessons learned from Running Hundreds of Kaggle Competitions: At Kaggle, we've run hundreds of machine learning competitions and seen over 80,000 data scientists make submissions. One thing is clear: winning competitions isn't random. We've learned that certain tools and methodologies work consistently well on different types of problems. Many participants make common mistakes (such as overfitting) that should be actively avoided. Similarly, competition hosts have their own set of pitfalls (such as data leakage).
In this talk, I'll share what goes into a winning competition toolkit along with some war stories on what to avoid. Additionally, I’ll share what we’re seeing on the collaborative side of competitions. Our community is showing an increasing amount of collaboration in developing machine learning models and analytic solutions. I'll showcase examples of this and discuss how these types of collaboration will improve how data science is learned and applied.
Kaggle presentation
Winning Kaggle competitions involves getting a good score as fast as possible using versatile machine learning libraries and models like Scikit-learn, XGBoost, and Keras. It also involves model ensembling techniques like voting, averaging, bagging and boosting to improve scores. The document provides tips for approaches like feature engineering, algorithm selection, and stacked generalization/stacking to develop strong ensemble models for competitions.
Kaggle Winning Solution Xgboost algorithm -- Let us learn from its author
This document provides an overview of XGBoost, an open-source gradient boosting framework. It begins with introductions to machine learning algorithms and XGBoost specifically. The document then walks through using XGBoost with R, including loading data, running models, cross-validation, and prediction. It discusses XGBoost's use in winning the Higgs Boson machine learning competition and provides code to replicate its solution. Finally, it briefly covers XGBoost's model specification and training objectives.
Tips for data science competitions
This document provides tips for winning data science competitions by summarizing a presentation about strategies and techniques. It discusses the structure of competitions, sources of competitive advantage like feature engineering and the right tools, and validation approaches. It also summarizes three case studies where the speaker applied these lessons, including encoding categorical variables and building diverse blended models. The key lessons are to focus on proper validation, leverage domain knowledge through features, and apply what is learned to real-world problems.
Dataiku at SF DataMining Meetup - Kaggle Yandex Challenge
This is a presentation made on the 13th August 2014 at the SF Data Mining Meetup at Trulia. It's about Dataiku and the Kaggle Personalized Web Search Ranking challenge sponsored by Yandex
Personalized Re-Ranking of Documents
This document discusses personalized search and re-ranking search results based on a user's profile and past behavior. It describes extracting features from query logs covering 27 days of search data to train a classifier. Features include documents clicked and time spent by both the same and different users for a given query. The model is trained using LambdaMART ranking algorithm on 24 days of data and validated on 3 days. It then re-ranks the top 10 search results for test queries based on the extracted features to provide a personalized search ranking. Evaluation on a test platform showed an NDCG score higher than the baseline, indicating more relevant results.
Quality Jam 2017: Jesse Reed & Kyle McMeekin "Test Case Management & Explorat...
Jesse Reed, QA Director at Questar, and Kyle McMeekin discuss how Questar made the switch to qTest and the key factors you should consider in test case management and exploratory testing.
Florian Douetteau @ Dataiku
The document describes a Kaggle competition to personalize web search results using historical search engine logs. A team called Dataiku used supervised learning techniques like random forests and LambdaMART gradient boosted trees to re-rank web search results for new queries based on features from past user search behavior, achieving a 1st place result. Their approach involved prototyping with fast methods, optimizing models with boosting, and being systematic about feature selection and validation.
Emptying Your Cup an Agile Primer
Bruce Nix provides an introduction to agile and Scrum. He summarizes his experience working with various companies to implement agile and Scrum practices. He then discusses some of the key principles of agile, including self-organizing teams, frequent delivery of working software, and responding to change. The rest of the document delves into the specifics of Scrum, including roles, ceremonies like the daily standup and sprint planning, and challenges with adoption.
Agile testing experiments
The document summarizes the author's experiments with agile testing practices at Yahoo. It discusses experiments with thinning the boundary between developers and testers, creating an automation framework using the same coding language for features and tests, using mind maps for test planning and reviews, implementing test huddles, and conducting group tests. The experiments aimed to improve quality, accelerate processes, and foster an agile mindset. The approaches that worked best were for new projects, small scrum teams, and co-located teams.
SenchaCon 2016: Creating a Flexible and Usable Industry Specific Solution - D...
Come hear how we used agile development and Sencha tools to meet our design requirements, to create a system that is highly configurable, flexible, and exceeds the expectations of our customers. Learn how to use an adaptive/responsive design to be able to support two very different types of users, with a single application, and using the same set of libraries. Most importantly, learn how to create a system that even your most important and demanding users will find value and actually want to use. By assembling some of the best programmers and DBA developers in the world, we have been able to create a best-in-class, fully functional, scalable and highly configurable system, while maintaining an amazingly easy-to-use interface.
Continuous Delivery without Significant Test Automation
All organisations aspiring agile don’t have a significant level of automated checks available. Deciding to start adding automation on top of legacy code is a long road to travel and while on route, releases will rely on more manual testing. Meanwhile, automating builds to a single click deploy already enables delivering.
This talk tells a story of going into mindset of continuous delivery of continuous releases to production without the level of test automation often seen as a key practice. You will learn from the changes we went through moving from monthly (scrum) to continuous (kanban) releases as we started delivering value in production in small incremental changes. The talk will help you see one route to a happy agile team with satisfied customers, where the recipe is built more around collaboration and smart exploratory testing than test automation to deliver a steady stream value.
My Experiments In Agile Testing in Yahoo.pptx
This document summarizes the author's experiments with agile testing practices at Yahoo. It describes experiments with thinning the boundary between developers and testers, developing an automation framework using the same language for tests and code, using mind maps for test planning and reviews, conducting daily test huddles, and performing group testing. The key lessons learned are that these practices can streamline testing, improve collaboration between teams, and help find issues earlier.
Play with Kaggle
The document discusses Kaggle, an online platform for predictive modeling competitions. It describes a dataset from Yandex containing query, click, and session data for evaluating search engine rankings. The goal is to build a model that can better predict click probabilities and sort URLs than Yandex's current algorithm. The proposed approach involves: 1) reshaping the data by user sessions; 2) cross-validating on a test set; 3) adding new user and query features; and 4) using logistic regression or random forests to predict click probabilities and improve the rankings.
Moving from fast to solr on atg
Presented by Ricardo Merizalde, Software Development Manager, Backcountry.com
The intent of this presentation is to describe an implementation of an open source framework for e-commerce sites. The presentation will focus on Oracle ATG but can be extended to other platforms. First, a brief introduction on what CommerceSearch is (an open source integration and framework for eCommerce sites). Review challenges we had with FAST Impulse. Review the main search artifacts merchandisers can manage through CommerceSearch. Review how CommerceSearch integrates with ATG to deploy changes in near real time. Review CommerceSearch integration test framework and automated test framework (Selenium). Finally, summarize the benefits we got by moving from FAST to Solr.
CommerceSearch: Moving from FAST to Solr on ATG
Presented by Ricardo Merizalde, Software Development Manager, Backcountry.com
The intent of this presentation is to describe an implementation of an open source framework for e-commerce sites. The presentation will focus on Oracle ATG but can be extended to other platforms. First, a brief introduction on what CommerceSearch is (an open source integration and framework for eCommerce sites). Review challenges we had with FAST Impulse. Review the main search artifacts merchandisers can manage through CommerceSearch. Review how CommerceSearch integrates with ATG to deploy changes in near real time. Review CommerceSearch integration test framework and automated test framework (Selenium). Finally, summarize the benefits we got by moving from FAST to Solr.
Agile Testing
dbg Agile Testing Presentation, demonstrating the use of Test Charters, Exploratory Testing, Session Based Testing and Testing Tours. With thanks to James Bach, Lisa Crispin, Janet Gregory and James Whittaker
20121213 qa introduction smileryang
This document discusses quality assurance in the software industry. It outlines various roles in a software project like project manager, developer manager, QA manager, and tester. It explains why software needs QA to test for issues from different perspectives. The document then describes common QA tasks like test planning, case creation, bug reporting, and verification. It provides details on testing methodologies like black box and white box testing. Examples of test case creation and bug reporting are illustrated. The document shares some stories of interactions between QA and other roles. It also discusses sources of happiness and unhappiness for those in QA roles.
User Centered Agile Development at NASA - One Groups Path to Better Software
The group at NASA iteratively adopted agile practices over two years to improve their software development process. They started with six month development cycles and shortened this to six weeks and then three weeks. This allowed for more frequent customer feedback and prioritization of work. Daily builds and testing ensured progress was always visible. Their measure of success became working code delivered in frequent iterations rather than presentations or documentation.
User centered agile dev balanced team 2013
The group at NASA iteratively adopted agile practices over two years to improve their software development process. They started with six month development cycles and shortened this to six weeks and then three weeks. This allowed for more frequent customer feedback and prioritization of work. Daily builds and testing ensured progress was always visible. Their measure of success became working code delivered in frequent iterations rather than presentations or documentation.
Agile by KD
The document provides an overview of the Agile methodology known as Scrum. It defines Scrum as an Agile process that allows teams to focus on delivering high business value in short iterations called sprints. The document outlines the key components of Scrum, including roles like Product Owner and Scrum Master, ceremonies like sprint planning and daily stand-ups, and artifacts like the product and sprint backlogs. It also discusses how Scrum teams self-organize to complete sprint goals and deliver working software incrementally in each sprint.
Agile by KD
The document provides an overview of the Agile methodology known as Scrum. It defines Scrum as an Agile process that allows teams to focus on delivering high business value in short iterations called sprints. The document outlines the key components of Scrum, including roles like Product Owner and Scrum Master, ceremonies like sprint planning and daily stand-ups, and artifacts like the product and sprint backlogs. It also discusses how Scrum teams self-organize to complete sprint goals and deliver working software incrementally in each sprint.
Journey of Migrating Millions of Queries on The Cloud
This document discusses challenges in upgrading a query engine and summarizing strategies for efficiently simulating queries to test compatibility and performance. It proposes grouping queries by signature and narrowing data scans to reduce the number of queries tested. It also recommends automating result verification by generating human-readable reports and excluding uncheckable queries. Assistance tools are proposed to aid investigation of differences, which helped discover real bugs in the target version.
SCQAA-SF Meeting on May 21 2014
This document provides information about the SCQAA-SF organization and an upcoming event. SCQAA-SF is a chapter that sponsors sharing of information to promote quality practices in IT through networking, training and professional development. They meet every two months in San Fernando Valley. The upcoming event will feature presentations on technology advancements and methodology, networking opportunities, and opportunities to earn PDU and other credits. Recently, the organization revised their membership dues policy to better accommodate members' needs.
Resume
Akansha Khanna seeks a challenging position applying her knowledge and experience. She has over 5 years of experience in testing for companies like Global Logic and Keysight Technologies. Her experience includes functional, regression, automation, and database testing. She is proficient with tools like Oracle CPQ, Salesforce, HPALM, SAS, and Excel. She has received excellence awards for her work and has experience leading teams and ensuring quality.
Scalability and performance for e commerce
Jerry Lewis, VP of an IBM practice, discusses website performance and scalability for eCommerce. He shares horror stories of performance issues causing major revenue losses and customer complaints. Website performance is important because slow sites hurt sales and customer experience. Common causes of bad performance include inefficient code, database issues, and third party integration problems. To achieve good performance, websites must be designed with performance in mind from the start, with strategies like caching, efficient database usage, and infrastructure tuning.
Effective Scrum
Scrum is a framework for managing product development that emphasizes iterative work cycles, frequent inspection points, and adaptation to change. It consists of sprints, daily stand-ups, sprint planning, reviews, and retrospectives. Benefits include increased responsibility, reduced risk, motivation, and continuous improvement. However, it can also be verbose with meetings and interrupt development flow if not implemented properly.
Java Certification by HUJAK - 2015-05-12 - at JavaCro'15 conference
Java Certification by HUJAK - 2015-05-12 - at JavaCro'15 conferenceHUJAK - Hrvatska udruga Java korisnika / Croatian Java User Association
This document provides biographies and credentials for several Java experts: Branko Mihaljević, Tihomir Smuđ, and Aleksander Radovan. It also discusses Java certification pathways including Oracle Certified Associate (OCA), Oracle Certified Professional (OCP), Oracle Certified Expert (OCE), and Oracle Certified Master (OCM) certifications. Details are given on certification exams, objectives, prerequisites, and example questions. Overall the document serves as an introduction to Java certification for developers.Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Discover the seamless integration of RPA (Robotic Process Automation), COMPOSER, and APM with AWS IDP enhanced with Slack notifications. Explore how these technologies converge to streamline workflows, optimize performance, and ensure secure access, all while leveraging the power of AWS IDP and real-time communication via Slack notifications.
More Related Content
Similar to What does it take to win the Kaggle/Yandex competition
Emptying Your Cup an Agile Primer
Bruce Nix provides an introduction to agile and Scrum. He summarizes his experience working with various companies to implement agile and Scrum practices. He then discusses some of the key principles of agile, including self-organizing teams, frequent delivery of working software, and responding to change. The rest of the document delves into the specifics of Scrum, including roles, ceremonies like the daily standup and sprint planning, and challenges with adoption.
Agile testing experiments
The document summarizes the author's experiments with agile testing practices at Yahoo. It discusses experiments with thinning the boundary between developers and testers, creating an automation framework using the same coding language for features and tests, using mind maps for test planning and reviews, implementing test huddles, and conducting group tests. The experiments aimed to improve quality, accelerate processes, and foster an agile mindset. The approaches that worked best were for new projects, small scrum teams, and co-located teams.
SenchaCon 2016: Creating a Flexible and Usable Industry Specific Solution - D...
Come hear how we used agile development and Sencha tools to meet our design requirements, to create a system that is highly configurable, flexible, and exceeds the expectations of our customers. Learn how to use an adaptive/responsive design to be able to support two very different types of users, with a single application, and using the same set of libraries. Most importantly, learn how to create a system that even your most important and demanding users will find value and actually want to use. By assembling some of the best programmers and DBA developers in the world, we have been able to create a best-in-class, fully functional, scalable and highly configurable system, while maintaining an amazingly easy-to-use interface.
Continuous Delivery without Significant Test Automation
All organisations aspiring agile don’t have a significant level of automated checks available. Deciding to start adding automation on top of legacy code is a long road to travel and while on route, releases will rely on more manual testing. Meanwhile, automating builds to a single click deploy already enables delivering.
This talk tells a story of going into mindset of continuous delivery of continuous releases to production without the level of test automation often seen as a key practice. You will learn from the changes we went through moving from monthly (scrum) to continuous (kanban) releases as we started delivering value in production in small incremental changes. The talk will help you see one route to a happy agile team with satisfied customers, where the recipe is built more around collaboration and smart exploratory testing than test automation to deliver a steady stream value.
My Experiments In Agile Testing in Yahoo.pptx
This document summarizes the author's experiments with agile testing practices at Yahoo. It describes experiments with thinning the boundary between developers and testers, developing an automation framework using the same language for tests and code, using mind maps for test planning and reviews, conducting daily test huddles, and performing group testing. The key lessons learned are that these practices can streamline testing, improve collaboration between teams, and help find issues earlier.
Play with Kaggle
The document discusses Kaggle, an online platform for predictive modeling competitions. It describes a dataset from Yandex containing query, click, and session data for evaluating search engine rankings. The goal is to build a model that can better predict click probabilities and sort URLs than Yandex's current algorithm. The proposed approach involves: 1) reshaping the data by user sessions; 2) cross-validating on a test set; 3) adding new user and query features; and 4) using logistic regression or random forests to predict click probabilities and improve the rankings.
Moving from fast to solr on atg
Presented by Ricardo Merizalde, Software Development Manager, Backcountry.com
The intent of this presentation is to describe an implementation of an open source framework for e-commerce sites. The presentation will focus on Oracle ATG but can be extended to other platforms. First, a brief introduction on what CommerceSearch is (an open source integration and framework for eCommerce sites). Review challenges we had with FAST Impulse. Review the main search artifacts merchandisers can manage through CommerceSearch. Review how CommerceSearch integrates with ATG to deploy changes in near real time. Review CommerceSearch integration test framework and automated test framework (Selenium). Finally, summarize the benefits we got by moving from FAST to Solr.
CommerceSearch: Moving from FAST to Solr on ATG
Presented by Ricardo Merizalde, Software Development Manager, Backcountry.com
The intent of this presentation is to describe an implementation of an open source framework for e-commerce sites. The presentation will focus on Oracle ATG but can be extended to other platforms. First, a brief introduction on what CommerceSearch is (an open source integration and framework for eCommerce sites). Review challenges we had with FAST Impulse. Review the main search artifacts merchandisers can manage through CommerceSearch. Review how CommerceSearch integrates with ATG to deploy changes in near real time. Review CommerceSearch integration test framework and automated test framework (Selenium). Finally, summarize the benefits we got by moving from FAST to Solr.
Agile Testing
dbg Agile Testing Presentation, demonstrating the use of Test Charters, Exploratory Testing, Session Based Testing and Testing Tours. With thanks to James Bach, Lisa Crispin, Janet Gregory and James Whittaker
20121213 qa introduction smileryang
This document discusses quality assurance in the software industry. It outlines various roles in a software project like project manager, developer manager, QA manager, and tester. It explains why software needs QA to test for issues from different perspectives. The document then describes common QA tasks like test planning, case creation, bug reporting, and verification. It provides details on testing methodologies like black box and white box testing. Examples of test case creation and bug reporting are illustrated. The document shares some stories of interactions between QA and other roles. It also discusses sources of happiness and unhappiness for those in QA roles.
User Centered Agile Development at NASA - One Groups Path to Better Software
The group at NASA iteratively adopted agile practices over two years to improve their software development process. They started with six month development cycles and shortened this to six weeks and then three weeks. This allowed for more frequent customer feedback and prioritization of work. Daily builds and testing ensured progress was always visible. Their measure of success became working code delivered in frequent iterations rather than presentations or documentation.
User centered agile dev balanced team 2013
The group at NASA iteratively adopted agile practices over two years to improve their software development process. They started with six month development cycles and shortened this to six weeks and then three weeks. This allowed for more frequent customer feedback and prioritization of work. Daily builds and testing ensured progress was always visible. Their measure of success became working code delivered in frequent iterations rather than presentations or documentation.
Agile by KD
The document provides an overview of the Agile methodology known as Scrum. It defines Scrum as an Agile process that allows teams to focus on delivering high business value in short iterations called sprints. The document outlines the key components of Scrum, including roles like Product Owner and Scrum Master, ceremonies like sprint planning and daily stand-ups, and artifacts like the product and sprint backlogs. It also discusses how Scrum teams self-organize to complete sprint goals and deliver working software incrementally in each sprint.
Agile by KD
The document provides an overview of the Agile methodology known as Scrum. It defines Scrum as an Agile process that allows teams to focus on delivering high business value in short iterations called sprints. The document outlines the key components of Scrum, including roles like Product Owner and Scrum Master, ceremonies like sprint planning and daily stand-ups, and artifacts like the product and sprint backlogs. It also discusses how Scrum teams self-organize to complete sprint goals and deliver working software incrementally in each sprint.
Journey of Migrating Millions of Queries on The Cloud
This document discusses challenges in upgrading a query engine and summarizing strategies for efficiently simulating queries to test compatibility and performance. It proposes grouping queries by signature and narrowing data scans to reduce the number of queries tested. It also recommends automating result verification by generating human-readable reports and excluding uncheckable queries. Assistance tools are proposed to aid investigation of differences, which helped discover real bugs in the target version.
SCQAA-SF Meeting on May 21 2014
This document provides information about the SCQAA-SF organization and an upcoming event. SCQAA-SF is a chapter that sponsors sharing of information to promote quality practices in IT through networking, training and professional development. They meet every two months in San Fernando Valley. The upcoming event will feature presentations on technology advancements and methodology, networking opportunities, and opportunities to earn PDU and other credits. Recently, the organization revised their membership dues policy to better accommodate members' needs.
Resume
Akansha Khanna seeks a challenging position applying her knowledge and experience. She has over 5 years of experience in testing for companies like Global Logic and Keysight Technologies. Her experience includes functional, regression, automation, and database testing. She is proficient with tools like Oracle CPQ, Salesforce, HPALM, SAS, and Excel. She has received excellence awards for her work and has experience leading teams and ensuring quality.
Scalability and performance for e commerce
Jerry Lewis, VP of an IBM practice, discusses website performance and scalability for eCommerce. He shares horror stories of performance issues causing major revenue losses and customer complaints. Website performance is important because slow sites hurt sales and customer experience. Common causes of bad performance include inefficient code, database issues, and third party integration problems. To achieve good performance, websites must be designed with performance in mind from the start, with strategies like caching, efficient database usage, and infrastructure tuning.
Effective Scrum
Scrum is a framework for managing product development that emphasizes iterative work cycles, frequent inspection points, and adaptation to change. It consists of sprints, daily stand-ups, sprint planning, reviews, and retrospectives. Benefits include increased responsibility, reduced risk, motivation, and continuous improvement. However, it can also be verbose with meetings and interrupt development flow if not implemented properly.
Java Certification by HUJAK - 2015-05-12 - at JavaCro'15 conference
Java Certification by HUJAK - 2015-05-12 - at JavaCro'15 conferenceHUJAK - Hrvatska udruga Java korisnika / Croatian Java User Association
This document provides biographies and credentials for several Java experts: Branko Mihaljević, Tihomir Smuđ, and Aleksander Radovan. It also discusses Java certification pathways including Oracle Certified Associate (OCA), Oracle Certified Professional (OCP), Oracle Certified Expert (OCE), and Oracle Certified Master (OCM) certifications. Details are given on certification exams, objectives, prerequisites, and example questions. Overall the document serves as an introduction to Java certification for developers.Similar to What does it take to win the Kaggle/Yandex competition (20)
SenchaCon 2016: Creating a Flexible and Usable Industry Specific Solution - D...
SenchaCon 2016: Creating a Flexible and Usable Industry Specific Solution - D...
Continuous Delivery without Significant Test Automation
Continuous Delivery without Significant Test Automation
User Centered Agile Development at NASA - One Groups Path to Better Software
User Centered Agile Development at NASA - One Groups Path to Better Software
Journey of Migrating Millions of Queries on The Cloud
Journey of Migrating Millions of Queries on The Cloud
Java Certification by HUJAK - 2015-05-12 - at JavaCro'15 conference
Java Certification by HUJAK - 2015-05-12 - at JavaCro'15 conference
Recently uploaded
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Discover the seamless integration of RPA (Robotic Process Automation), COMPOSER, and APM with AWS IDP enhanced with Slack notifications. Explore how these technologies converge to streamline workflows, optimize performance, and ensure secure access, all while leveraging the power of AWS IDP and real-time communication via Slack notifications.
A tale of scale & speed: How the US Navy is enabling software delivery from l...
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
UiPath Test Automation using UiPath Test Suite series, part 6
Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI.
UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities.
Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes.
What will you get from this session?
1. Insights into integrating generative AI.
2. Understanding how this integration enhances test automation within the UiPath platform
3. Practical demonstrations
4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath
Topics covered:
What is generative AI
Test Automation with generative AI and Open AI.
UiPath integration with generative AI
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Pushing the limits of ePRTC: 100ns holdover for 100 days
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
Building RAG with self-deployed Milvus vector database and Snowpark Container...
This talk will give hands-on advice on building RAG applications with an open-source Milvus database deployed as a docker container. We will also introduce the integration of Milvus with Snowpark Container Services.
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing.
One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.
Microsoft - Power Platform_G.Aspiotis.pdf
Revolutionizing Application Development
with AI-powered low-code, presentation by George Aspiotis, Sr. Partner Development Manager, Microsoft
20240607 QFM018 Elixir Reading List May 2024
Everything I found interesting about the Elixir programming ecosystem in May 2024
Full-RAG: A modern architecture for hyper-personalization
Mike Del Balso, CEO & Co-Founder at Tecton, presents "Full RAG," a novel approach to AI recommendation systems, aiming to push beyond the limitations of traditional models through a deep integration of contextual insights and real-time data, leveraging the Retrieval-Augmented Generation architecture. This talk will outline Full RAG's potential to significantly enhance personalization, address engineering challenges such as data management and model training, and introduce data enrichment with reranking as a key solution. Attendees will gain crucial insights into the importance of hyperpersonalization in AI, the capabilities of Full RAG for advanced personalization, and strategies for managing complex data integrations for deploying cutting-edge AI solutions.
Introducing Milvus Lite: Easy-to-Install, Easy-to-Use vector database for you...
Join us to introduce Milvus Lite, a vector database that can run on notebooks and laptops, share the same API with Milvus, and integrate with every popular GenAI framework. This webinar is perfect for developers seeking easy-to-use, well-integrated vector databases for their GenAI apps.
National Security Agency - NSA mobile device best practices
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
Climate Impact of Software Testing at Nordic Testing Days
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Communications Mining Series - Zero to Hero - Session 1
This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered:
• Communication Mining Overview
• Why is it important?
• How can it help today’s business and the benefits
• Phases in Communication Mining
• Demo on Platform overview
• Q/A
Large Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial Applications.
Video Streaming: Then, Now, and in the Future
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
Recently uploaded (20)
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
Building RAG with self-deployed Milvus vector database and Snowpark Container...
Building RAG with self-deployed Milvus vector database and Snowpark Container...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
Introducing Milvus Lite: Easy-to-Install, Easy-to-Use vector database for you...
Introducing Milvus Lite: Easy-to-Install, Easy-to-Use vector database for you...
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
Large Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial Applications
What does it take to win the Kaggle/Yandex competition
- 1. WHAT DOES IT TAKE TO WIN THE KAGGLE/YANDEX COMPETITION Christophe Bourguignat Kenji Lefèvre-Hasegawa Paul Masurel @Dataiku Matthieu Scordia @Dataiku
- 2. OUTLINE OF THE TALK • Review of the Kaggle/Yandex Challenge • How we worked (team work & tools) • The winning model
- 3. GOAL Re-rank URLs returned by Yandex according to the personal preferences of the users url1 url3 url2 url2 GOAL url3 url1 url4 url4 ML CHALLENGE Predict user’s pertinence for urls and rerank result set accordingly The Kaggle/Yandex challenge
- 4. GIVEN • 30 days logs test: 3 days, train: 27 days • Users historic queries, clicks & dwell-times Q Q Q Q • Test session prior activity queries, clicks & dwell-times Test session : SIZE • 15Gb size The Kaggle/Yandex challenge Q Q T ?
- 5. QUALITY METRIC • One query test / user on the last 3 days • NDCG metric penalize error of pertinence on top ranked urls • No A/B test url1 url2 OK BAD url4 url3 Kaggle The Kaggle/Yandex challenge Prediction Another ranking
- 6. TEAM DATAIKU SCIENCE STUDIO / KAGGLE • • • • Christophe Bourguignat Engineer, Data enthusiastic Kenji Lefèvre-Hasegawa Ph.D. math, new to ML Paul Masurel Software Engineer @dataiku Matthieu Scordia Data Scientist @dataiku First meeting : October16th 2013 How we worked (Team work & tools)
- 7. WE’VE USED • • • • • Related papers (mainly Microsoft’s) 12 core, 64 Gb Python scikit-learn Dataiku Science Studio Java Ranklib How we worked (Team work & tools)
- 8. DATAIKU SCIENCE STUDIO Features & labels Features Labels Split train & validation Original train LEARNING Team members work independantly FEATURES CONSTRUCTION Team members work independantly DATA DRIVEN COMPUTATION How we worked (Team work & tools)
- 9. HOW MUCH WORK ? • 960+ emails • 360+ features • 50+ ideas grid tuned (300+ models fitted) • Server heavily loaded the last 3 weeks • 56 kaggle submissions • 196 teams, 264 players, 3570 submissions How we worked (Team work & tools) 2014-01-01 1st Future top 2 & 3 enter race 1 week 3rd 1 week 1st 5th Top 10 Top 25 1/2 month 1 week
- 10. PROBLEM ANALYSIS Query Result Set • Rank • URL Snippet Quality • URL is skipped, clicked or missed CLICK Reading URL • URL & Domain pertinence with dwell-time The winning model
- 11. FEATURES Features : • Rank • User habits, query specificity (entropy, frequency,…) • Snippet pertinence • Missed, Skipped, Clicked • URL & Domain Pertinence Declinaison of & Clicked • Probability, Stimuli freq., Mean Reciprocal Rank (MRR) • For each user : historic & previous activity in test session & aggregate • For all user • Declined for all queries & same query The winning model
- 12. MODELS • Random Forest (predict proba) + maximize E(NDCG) Kaggle/Yandex Top 1 then 3rd • Lambda MART (Gradient Boosting Tree optimized for NDCG) WINS ! The winning model