Your SlideShare is downloading. ×
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Test articles 2009 to 2013
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Test articles 2009 to 2013

7,001

Published on

Compilation of test articles I have read and sent out as a news mail between 2009-2013

Compilation of test articles I have read and sent out as a news mail between 2009-2013

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
7,001
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
38
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Test Articles Compilation from 2009- Oct to 2013-Aug by Johan Hoberg Introduction This document is a compilation of test articles ( or articles relevant to testing) I read and sent out as a news mail during 2009-2013. The idea of this document is to use it when you are searching for information, find the relevant articles and then go and read those articles from the original source. When I started in 2009 I did not have this in mind, and did not copy parts of the text from the original source of the article, which makes those articles more difficult to search unless the topic is in the title/link text, since those first article links are mostly the actual link and maybe a title, while later articles will have a small part of the article copied with the title and the link. The first articles are at the end of the document, and the latest articles are in the beginning, so the most relevant articles are quick to find. Test Articles 2009-2013 Fake your way to better tests http://googletesting.blogspot.se/2013/06/testing-on-toilet-fake-your-way-to.html ”After many years of blogging, you decide to try out your blog platform's API. You start playing around with it, but then you realize: how can you tell that your code works without having your test talk to a remote blog server? Fakes to the rescue! A fake is a lightweight implementation of an API that behaves like the real implementation, but isn't suitable for production.” Know your test doubles http://googletesting.blogspot.se/2013/07/testing-on-toilet-know-your-test-doubles.html ”A test double is an object that can stand in for a real object in a test, The most common types of test doubles are stubs, mocks, and fakes. A fake doesn’t use a mocking framework: it’s a lightweight implementation of an API that behaves like the real implementation, but isn't suitable for production (e.g. an in-memory database).” Thoughts about automation from Microsoft http://angryweasel.com/blog/?p=652
  • 2. “Automation” is an overloaded and abused word that often implies that test code only exists to automate human tasks – that’s stupid If you are a tester who codes, you write code. Period. Some of your code may automate tasks. Most will not (or shouldn’t) “The point is, if your testing strategy is entirely to automate a suite of user tasks, you are doing way more to keep “automators” busy than you are actually providing valuable testing.” Optimal Logging http://googletesting.blogspot.se/2013/06/optimal-logging.html How long does it take to find the root cause of a failure in your system? Five minutes? Five days? If you answered close to five minutes, it’s very likely that your production system and tests have great logging. All too often, seemingly unessential features like logging, exception handling, and (dare I say it) testing are an implementation afterthought. Smart Monkey http://experttesters.com/2013/06/04/theres-a-smart-monkey-in-my-toolbelt/ Dumb Monkey testing is when the automation randomly sends keystrokes or commands to the System Under Test (SUT). Smart Monkey testing is, well, smarter. Instead of doing random things, the automation system knows something about your program: Where it is What it can do there Where it can go and how to get there Where it’s been If what it’s seeing is correct Top Secrets from Top Testers http://blog.utest.com/introducing-the-top-secrets-of-top-testers/2013/06/ As part of a new series that we’re calling The Top Secrets of Top Testers, we’ll be asking a panel of testing experts, uTest community members and internal team members a new question every month. As you might have guessed, this month’s question is: “If you could offer a new tester one piece of advice, what would it be?” Coding Mindset http://angryweasel.com/blog/?p=649 This mindset, although common, kills me: Testers who code write automation Automation is primarily made up of a suite of user tasks Automation is primarily done through the GUI All of these items are, in my opinion, idiotic – and likely harmful to software quality. It focuses test teams in the wrong place and is, frankly, insulting to testers who know coding. If you’re a tester who writes code (or a coder who cares about testing), try this: Write tools that help you find important bugs quickly
  • 3. Write a few high level tests that ensure user actions “work”, but leave most of it for exploratory testing, user testing (assuming you have great monitoring tools), or both. If you must write GUI tests for the above, write fewer of them (that way you’ll spend less time investigating their flaky behavior). If you’re a tester who writes code, you’re not a test automator, you’re not a test automation engineer, and you don’t write test automation or test code. You write code. And you test. Modeling Programmer Productivity http://secretsofconsulting.blogspot.se/2013/05/modeling-programmer-productivity.html "There is a perception in some tech circles that older programmers aren’t able to keep pace with rapidly changing technology, and that they are discriminated against in the software field. But a new study from North Carolina State University indicates that the knowledge and skills of programmers actually improve over time—and that older programmers know as much (or more) than their younger peers when it comes to recent software platforms." ”This study didn't surprise me. Nor did it surprise many of my programmer friends. Why, then, do so many managers still believe that older programmers are less skilled than their younger co-workers?” State Transition Testing http://experttesters.com/2013/05/08/state-transition-testing/ State Transition Diagrams, or STDs as they’re affectionately called, are effective for documenting functionality and designing test cases. They should be in every testers bag of tricks, along with Decision Tables, Pair-Wise analysis, and acting annoyed at work to appear busy. One benefit of State Transition Diagrams is that they describe the behavior of the system in a complete, yet easy-to-read and compact, way. Often, the best strategy is to create tests that cover all transitions (the arrows) at least once. This guarantees you will test every state, event, action, and transition. It gives you good coverage in a reasonable amount of tests. Metric Driven Development http://www.infoq.com/articles/metrics-driven-development So what is MDD? I would define MDD as a practice where metrics are used to drive the entire application development. In a company which uses MDD, everything from performance and usage patterns to revenue is measured. Moreover, every single decision taken by developers, operations or even business people is based on metrics. Metrics are used to monitor team performance, solve performance bottlenecks, estimate hardware needs or to accomplish other purposes at any stage of development life-cycle. Asshole Driven Development http://scottberkun.com/2007/asshole-driven-development/ The software industry might be the world’s greatest breeding ground for new systems of management. From Agile, to Extreme Programming , to Test Driven Development (TDD), the acronyms and frameworks keep piling up. Why? Some say it’s immaturity: that software is still a young industry and all the change is the path to some true fundamentals. Others say it’s because software people like making things up and can’t help
  • 4. themselves. Well I say this: if we’re going to have dozens of models we may as well have some that are honest, however cynical, to what’s really going on much of the time. Tear down the Wall http://angryweasel.com/blog/?p=624 There’s a wall between testers and programmers. Programmers “build”, testers “break” – but passing features back and forth between one group and another is wasteful. It takes time and extra communication to traverse the wall. What software teams need in the future is team members who can perform the activities of programming, testing, and analysis – together. I saw a slide deck recently that stated, “Agile Testers frequently can’t keep up with Programmers”. Good software development can’t happen when you serialize tasks – the team needs to work on this stuff – together. I’ll always be deeply involved in the activity of software testing, but I don’t know if the role or title exists in my future. After nearly 20 years of being a tester (and likely several more with that title), I’m admittedly going out on a bit of a limb with that statement. Despite my title, I want the walls to go away. GTAC (Google Test Automation Conference) 2013 Slides and Presentations https://developers.google.com/google-test-automation-conference/2013/presentations Topics such as Android UI Automation How Facebook tests Facebook on Android How do you Test a Mobile OS? Finding Culprits automatically in failing builds, i.e. Who broke the Build? Evolution from Quality Assurance to Test Engineering Escape from Brainstorm Island http://blogs.hbr.org/video/2013/04/escape-from-brainstorm-island.html Innovation is not about specific innovation events or workshops – it is a continuous work which everyone needs to be involved in on a daily basis. Emphasize Good Practice http://www.noop.nl/2013/05/emphasize-good-practices.html However, I noticed a strange thing when I urged people to stop screwing up. I found this didn’t motivate them at all! I realized getting better isn’t just about reducing what goes wrong (making mistakes). It’s also about increasing what is right (using good practices). And every now and then people need a reminder that they’re doing just fine. Test Management is Wrong (Don’t forget to read the comments, which are equally or more valuable than the article) http://swtester.blogspot.com.au/2013/02/test-management-is-wrong.html Basically, don’t get so bogged down in test management (the process) that you overlook actual quality. Checking boxes on your testing to-do list isn’t the same thing as working to make sure software is actually of high quality – bug free (at least mostly), efficient, effective and achieving its goal. Testing the Limits with Fiona Charles
  • 5. http://blog.utest.com/testing-the-limits-with-fiona-charles/2013/04/ How to rescue a failing test project: Interview a range of people to establish the facts and perceptions. Set a clear direction and make sure everyone is working productively on tasks that are aligned with it. Dig for and lay out the facts about the system quality or project process issues, and report the findings in a blame-free way. Make sure there’s a test strategy oriented to the stakeholders’ values and the business risks and a viable plan to execute on it. Fill any skills gaps on the team. Help build communication bridges and coach the test team to develop working relationships across the project. Make sure every tester has good reason to feel empowered and valued for his/her contribution. Kanban and Scrum – Making the most of both https://open.dsv.su.se/pluginfile.php/1483/mod_resource/content/1/KanbanAndScrumInfoQVersionFI NAL.pdf Book about differences and similarities between Kanban and Scrum and how they complement and conflict with each other. Published 2010. Testing State vs. Testing Interaction http://googletesting.blogspot.se/2013/03/testing-on-toilet-testing-state-vs.html There are typically two ways a unit test can verify that the code under test is working properly: by testing state or by testing interactions. What’s the difference between these? That Damn Gorilla http://angryweasel.com/blog/?p=594 Last week I happened to log on to twitter just as some test folks were marveling over the the massive parallels between the video with the gorilla and the basketball players and software testing. I made some cracks about the absurdity of the hype, and more than a few testers freaked out. Software Testing is like … http://blog.utest.com/software-testing-is-like/2013/03/ When you work in technology, you will inevitably have to explain your job to non-technical people. This isn’t always easy (and it’s certainly not much fun). Worse yet, most the time we don’t give ourselves enough credit when explaining our profession to friends and relatives; we sell ourselves short. Purpose over targets http://www.noop.nl/2013/03/purpose-over-targets.html That’s the difference between a purpose and a target. And that’s why I don’t like how many apply the SMART model for goals, because the SMART model moves people’s attention away from a purpose to a target. Your targets are useless if you can’t identify your purpose. Samsung trying out 3 CEOs http://blogs.hbr.org/cs/2013/03/samsung_tests_whether_three_he.html The announcement last week that the Samsung Electronics is elevating two executives, Boo-Keun Yoon and J. K. Shin, to the CEO role was met with interest in leadership circles. We have seen this kind of co-CEO arrangement grow over the years. All the more interesting: the South Korean giant's
  • 6. current CEO and vice chairman Oh-Hyun Kwon isn't even vacating the seat. All three men will now share the role. Testing vs. Checking Redefined – by James Bach http://www.satisfice.com/blog/archives/856 Testing is the process of evaluating a product by learning about it through experimentation, which includes to some degree: questioning, study, modeling, observation and inference. Human checking is an attempted checking process wherein humans collect the observations and apply the rules without the mediation of tools. Machine checking is a checking process wherein tools collect the observations and apply the rules without the mediation of humans. Human/machine checking is an attempted checking process wherein both humans and tools interact to collect the observations and apply the rules. Automated vs. Exploratory – we got it wrong! http://www.shino.de/2013/03/21/automated-vs-exploratory-we-got-it-wrong/ ”There is a misconception floating around that Agile Testing is mostly about automated checking. On the other way the whole testing community fights since decades about the value of test automation vs. the value of exploratory testing. I find these two different extreme position unhelpful to do any constructive testing. Most recently I ended up with a model that explains the underlying dynamics quite well, to testers, to programmers, and to their managers. Here’s the model.” Why organizations are so afraid to simplify http://blogs.hbr.org/ashkenas/2013/03/why-organizations-are-so-afraid-to-simplify.html ”It's one of the great contradictions of organizational life: People are great at starting new things — projects, meetings, initiatives, task forces — but have a much harder time stopping the ones that already exist.” The Perimeter Test http://enjoytesting.blogspot.se/2013/03/the-perimeter-test-new-test-idea.html “I want to include a test to click around the button to check if the focus of the button is restricted to the button area or extends outside the button area. According to me, the name for the test idea: "The Perimeter Test".” Good Requirements Deliver a high ROI http://blog.enfocussolutions.com/Powering_Requirements_Success/bid/172503/Good-Requirements- Deliver-a-High-ROI ”A while back, Ellen Gottesdiener from EBG Consultants assembled a list of citations about ROI and requirements titled Do You Know Your ROI on Your Requirements Work. Ellen did an excellent job researching this topic and presents a comprehensive list of citations in her article.” Management Myth 15: People need to work overtime http://www.stickyminds.com/sitewide.asp?Function=edetail&ObjectType=ART&ObjectId=17952&tt h=DYN&tt=siteemail&iDyn=2
  • 7. ”When managers ask people to work overtime, they are not thinking about what a technical day is like. Especially if you work on an agile team, they are not thinking of what an agile day is like. A manager’s day is fraught with interruption and context switching every ten minutes. Many managers feel as if they need to wait until 5 p.m. to get anything done. But if you are a technical person who has arranged your day so you have a good two-hour chunk in the morning, and a couple of more two-hour chunks in the afternoon, you have accomplished your work for the day. Your brain is tired and you need to leave. If you have paired or swarmed or worked with a cross-functional team, you are tired. You need to leave, to refresh your brain for the next day.” Stop Blaming the System http://www.noop.nl/2013/03/stop-blaming-the-system.html ”I see it again and again. The suggestion of systems thinkers and Agile writers to stop blaming people and instead try blaming the system. Seriously, what is the difference between blaming other people versus blaming the system? It seems to me it’s all the same, as long as we’re not blaming ourselves!” Why programming classes need to cover software testing too http://www.networkworld.com/community/node/82579 ”Software testing has been on a path toward convergence with programming for some time, especially when it comes to test automation. At the same time, many vendors have tried to create tools that allow non-programmers to exercise code in ways they would not know how to do by themselves.” ”At the same time, software testers provide much-needed input and perspective from exploratory testing and user acceptance testing. In my opinion, nothing can replace the “human perspective” as part of the quality assurance cycle.” The Meaning of Models http://www.noop.nl/2013/03/the-meaning-of-models.html ”There were several other documents with boxes and arrows, definitions and categories, quadrants and matrices, circles and pyramids, and they didn’t mean much to me. I’m a bit tired of such models, I noticed. I’ve probably seen too many.” High Level Overview of Software Testing http://thetesteye.com/blog/2013/03/open-letter-to-eurostar-organizers-testing-introduction/ ”Thanks for your request of a high level summary of software testing. You would get different answers from each tester, and here’s what I think you should know.” The 60% Rule http://www.noop.nl/2013/03/the-60-percent-rule.html ”I’ve heard some coaches suggest a 60% rule: everyone should spend 60% of their time on the same team, the same department, or the same organization. The rest of their time they can spend working for others.” An Overview of High Volume Automated Testing http://kaner.com/?p=278 ”This overview describes the general concept of high volume automated testing (HiVAT) and twelve examples of HiVAT techniques. The common thread of these techniques is automated generation, execution and evaluation of arbitrarily many tests. The individual tests are often weak, but taken together, they can expose problems that individually-crafted tests will miss. The simplest techniques offer high coverage: they can reach a program’s weakness in handling specific special cases. Other
  • 8. techniques take the program through long sequences of tests and can expose programs that build gradually (e.g. memory leaks, stack corruption or memory corruption) or that involve unexpected timing of events. In my experience, these sequences have exposed serious problems in embedded software, especially in systems that involved interaction among a few processors. As we embed more software into more consumer systems, including systems that pose life-critical risks, I believe these tests will become increasingly important.” Checklist for Software Testing Outsourcing http://www.kaner.com/pdfs/oursourcing1997.pdf ”When lawyers negotiate a contract, they often find it handy to work from a checklist of key issues. Fewer things slip through the cracks, and the negotiator doesn’t have to rethink the same issues in every contract. Further, the checklist grows and cumulates experience.” Being an Effecivte Spec Reviewer (Microsoft) http://experttesters.com/2013/02/28/being-an-effective-spec-reviewer/ ”The first time testers can impact a feature is often during functional and design reviews. This is also when we make our first impression on our co-workers. If you want to make a great initial impact on both your product and peers, you have to be an effective reviewer” Stakeholder Matrix for Stakeholder Analysis http://businessanalyst.tarunchandel.com/2012/10/stakeholder-matrix-for-stakeholder.html A stakeholder matrix, like the one in the above diagram, is a great tool to identify the stakeholders and also classify them appropriately. The classification based on influence of and impact of the change on stakeholder, helps to reduce the number of stakeholders to a manageable size. 6 Testing Mistakes to Avoid http://blog.utest.com/6-testing-mistakes-to-avoid-like-the-plague/2013/02/ Mistake #1: Testing too late Mistake #2: Testing with amateurs Mistake #3: Testing without a scope Mistake #4: Testing “one and done” Mistake #5: Testing in a controlled environment Mistake #6: Testing too fast/slow Black Box vs. White Box http://thetesteye.com/blog/2013/02/black-box-vs-white-box/ ”I have heard and seen the difference between Black Box Software Testing and White Box Software Testing being described as “whether or not you have knowledge about the code”; and that Gray Box Software Testing is a mix of the two. But is it really about how much of the code you see?” Value Creation over Value Extraction http://www.noop.nl/2013/02/value-creation-over-value-extraction.html ”But new value can only be created when that which is already valuable remains intact. When you delight customers while screwing suppliers, you’re not creating value, you’re just moving it from one stakeholder to the other. When you increase productivity while cutting corners in quality, you’re also not creating value, you’re just stealing it from the future.” The Insapience of Anti-Automationism
  • 9. http://context-driven-testing.com/?p=69 ”The distinction between manual and automated is a false dichotomy. We are talking about a matter of degree (how much automation, how much manual), not a distinction of principle.” Don’t let Scrum make you Fragile http://www.noop.nl/2013/02/dont-let-scrum-make-you-fragile.html ”A complex system benefits from not following the same practices over and over again. By adding a little bit of stress, and continuous variability in the environment, the system learns to become fit and healthy.” Firmware vs. Software http://www.renaissancesoftware.net/blog/archives/458 ”I have nothing against firmware, or firmware engineers (I’ve been known to write some firmware myself). But what we really need is less firmware and more software. Actually, I am disappointed that firmware engineers write so much firmware!” Should business people care about clean code? http://www.renaissancesoftware.net/blog/archives/476 ”Do business people care about clean code? They behave like they don’t. Well, they do and I can prove it using proof by contradiction. Suppose that a business person does not care about clean code. The business person wants features and they want them fast. So they pressure engineers to just get the code working. This causes the code to deteriorate. The business person wants more features, but the code is a mess so features take longer to add and things that work start to break. If the business person wants features fast, they need clean code and if they need clean code they must care about it too, which contradicts the statement that business people don’t care about clean code.” Exploratory Testing = Fun Productivity http://experttesters.com/2013/02/07/exploratory-testing-fun-productivity/ ”Exploratory testing can be valuable in specific situations and reveal certain categories of defects more readily than other approaches. The overall effectiveness of behavioral testing approaches is heavily influenced by the tester’s in-depth system and domain knowledge and experience. Of course, the effectiveness of any test method eventually plateaus or becomes less valuable and testers must employ different approaches to further investigate and evaluate the software under test” Please reproduce this issue http://angryweasel.com/blog/?p=573 “… how many times have you seen a test error, and immediately re-run the test to see if you could reproduce the error yourself? Is it a big number? If it is, you’re not going to like what I have to say. If you need to reproduce all of your bugs to figure out what’s going on, you screwed up. Your logging is bad. Your diagnostics don’t exist. You wrote crummy code. You’re wasting time!” Double Testing http://thetesteye.com/blog/2013/02/double_testing/ ”I think the idea of double testing is about what models we use in testing. More precisely our mental models on our test approach/perspective on testing, system boundaries, system parts, levels of testing, terminology in testing, ideas of test coverage, usage of test techniques, test idea sources, test planning techniques and so on.” The Merit System http://www.noop.nl/2013/02/the-merits-system.html
  • 10. “It is clear that the annual bonus system doesn't work. A tremendous amount of research says that it demotivates people, destroys collaboration, and causes dysfunctional behavior among managers and employees. What also doesn't work is the flat system, where everyone simply gets the same compensation or bonus. This also demotivates most people. And it makes organizations inflexible in times when agility is needed. What we need is a merits system...” Root Cause Analysis Template http://www.allthingsquality.com/2013/01/root-cause-analysis-template.html “When something gets released to Production which adversely affects a customer, we sometimes write up a formal Root Cause Analysis to send to the customer. It basically explains what happened, and how we intend to prevent it from happening again.” Future of Testing http://angryweasel.com/blog/?p=569 ”Regardless of how you feel about the health of software testing (which depends largely on your ability to interpret a metaphor), for me, it’s getting easier and easier to see that testing is changing. Granted, it’s changing along with the software under test, so if you’re testing the same sort of desktop software products you have been for years, using development practices even older, the good news is that you’re safe – your testing world probably isn’t going to change either.” Test Engineers @ Google http://googletesting.blogspot.se/#!/2013/01/test-engineers-google.html ”In order to get a clearer picture of what Test Engineers are responsible for, I chatted with three of my colleagues. We were able to identify the underlying Test Engineers’ similarities, while highlighting the differences.” How to recognize a problem http://www.developsense.com/blog/2013/01/oracles-the-brainstream-media/ “So: sometimes as we’re testing, we recognize a problem by a direct feeling, or through a principle that we’re aware of. Sometimes our recognition of a problem comes from a document or tool; sometimes from another person. That may entail embodied mental processes, rational principles, or mechanisms, but all that’s kind a mouthful. So lately, James and I have started to say that an oracle is simply “a way to recognize a problem”.” Debugging for Testers http://angryweasel.com/blog/?p=565 “I came across a few comments and statements recently that set off a slightly red (orange?) flag for me. The gist was that debugging was for coders, and that testers “just found the issues”. I get the context (or at least I think I get it) – I agree that all testers don’t have to be coders, and that all testers certainly don’t have to be debugging masters, but I don’t think that any testers should run away from a debugger in fear either.” Guide to Technical Testing http://www.slideshare.net/eviltester/evil-testers-guide-to-technical-testing Presentation about what a technical tester is. Managing your requirements https://www.ibm.com/developerworks/mydeveloperworks/blogs/requirementsmanagement/entry/why- requirements-management?lang=en
  • 11. https://www.ibm.com/developerworks/mydeveloperworks/blogs/requirementsmanagement/entry/good -requirements-writing?lang=en ”The world of requirements management has developed significantly in the last decade or so and has increasingly become one of the corner stones of successful software and systems engineering projects. We have been discussing various aspects of the domain from a best practices perspective and how tools can help managing your requirements efficiently and effectively. Starting today we will discuss various aspects of the requirements management discipline at a bird’s eye view level. These are meant to be introductory in nature and also intend to serve as refreshers for those who are already in the field.” [Swedish Article] Tala är Silver, Skriva är Guld http://kravexperten.se/tala-ar-silver-skriva-ar-guld/ ”När det agila manifestet ses som universallösning på systemutvecklingens utmaningar har vi en tendens att slänga ut barnet med badvattnet och övertolka principer som ”fungerande programvara framför omfattande dokumentation”. Omfattande dokumentation vantolkas inte sällan som ”så gott som ingen dokumentation – i alla fall inte mer än vad jag behöver nu” och plötsligt har vi dragit undan mattan för möjligheten att kommunicera bakomliggande behov och krav i skriftlig form över lång tid.” Scaled Agile Framework http://scaledagileframework.com/ This is a model for implementing Agile on an enterprise level. This is not new, but was mentioned in the article below. I thought it was a nice overview. Agile at VersionOne http://managetotest.com/2012/12/13/agile-at-versionone/ Continuous integration is an important part of the process. They use Jenkins to manage the builds for each team, tracking both checkin and nightly test runs which comprise both unit, and acceptance tests. They are currently iterating through the use of cucumber to create scenario tests. Steve reports that they haven’t had much success at using the tests as “living documentation” but they have seen value in the collaboration it fosters in helping refine stories. I asked about UI automation, and they do have some, but mostly used for high level scenario and process flow checks. After a recent STPCon, the lone dedicated tester of the group discovered the glories of Session-Based Testing. This method of managing Exploratory Testing is quickly being adopted by the teams. Rules I’ve Learned and Live by to Motivate People and Organizations http://www.linkedin.com/today/post/article/20121129185100-84166298-rules-i-ve-learned-and-live- by-to-motivate-people-and-organizations http://www.linkedin.com/today/post/article/20121210172831-84166298-my-rule-2-for-motivating- people-and-organizations http://www.linkedin.com/today/post/article/20121217200430-84166298-rule-3-for-motivating-people- and-organizations http://www.linkedin.com/today/post/article/20130107033123-84166298-learn-about-your-managers- through-the-people-they-manage It is your career – Lead it! (Microsoft) http://testastic.wordpress.com/2012/10/10/its-your-career-lead-it/ No review system is perfect and it turns out the Peter Principle (“people get promoted to the point of incompetency”) is true. It was proven in 2010 by the Nobel Peace prize winners of Management
  • 12. Science (here). Microsoft follows a general “Up or Out” strategy, which is considered a typical “defense” to Peter Principle happening in your org. These same guys proved that promoting the most competent people from the lower levels up is not as successful of a strategy as alternating between the Best and Worst when promoting. Organisational Culture Defined (Old, but a good overview of organisational culture) http://www.sidewaysthoughts.com/blog/2010/11/organisational-culture-defined-courtesy-of-edgar- schein/ I hear people referring to the culture of a place as being “good” or “bad”, managers and consultants speak of changing culture, and employees speak of being part of culture. What is this culture they all refer to? My studies introduced me to Edgar Schein’s answer, and confirm that the task of getting culture “right” is a tough target to hit. Risk Based Testing – Two parts http://blog.utest.com/guest-post-intro-to-risk-based-testing/2012/11/ ”Now, when we test a product with a limited number of testers in a small amount of time, we have to curtail the amount of testing that can be performed. This is where the concept of Risk Based Testing comes in. Let us understand the concept in detail.” http://www.webapptesting.com/guest-post-risk-based-testing-102-2/2012/12/ The RBT process can be described by these major steps: Step1 – Define all requirements in terms of Risk associated to them Step2 – Based on risk assessment, prioritize the requirements Step3 – Plan and define tests according to requirement prioritization Step4 – Execute test according to prioritization and acceptance criteria. The Most important advantages of this approach are as follows: Linking the product risks to the requirements identifies gaps. With limited time, money and qualified resources, testing concentrates on the most important matters first, thus delivering the most optimal test. The focus lies on risks to the business instead of just the functionality of the information system. During testing, communication (Test Reporting) takes place in a language (risks) that all stakeholders understand, hence involvement of testing is greater in comparison to mere list of issues communicated. It offers a negotiating instrument to client and test manager alike when available means are limited. Experts say it is time to give Java the boot http://www.pcworld.com/article/261843/time_to_give_java_the_boot_.html ”Java, the programming language designed to make the web fun and interactive, has become one of the weakest links in a PC’s and Mac’s defenses against external threats. Consider the most recent Java vulnerability, a weakness currently being exploited by malware distributors: When Oracle, Java’s maker, released an emergency update to fix the software, security analysts reported that even the hot-off-the-presses code contains additional vulnerabilities.” Decision Table Testing http://experttesters.com/2012/12/03/decision-table-testing/
  • 13. “Decision tables are an effective tool for describing complex functional requirements and designing test cases. Yet, although they’re far from a new concept, I’ve only seen a handful of functional specs and test plans that included one.” What’s Comparable – Part 1 http://www.developsense.com/blog/2012/12/whats-comparable-part1/ “People interpret requirements and specifications in different ways, based on their models, and their past experiences, and their current context. When they hear or read something, many people tend to choose an interpretation that is familiar to them, which may close off their thinking about other possible interpretations. That’s not a big problem in simple, stable systems. It’s a bigger problem in software development. The problems we’re trying to solve are neither simple nor stable, and the same is true with the software that we’re developing.” The Kudo Box http://www.noop.nl/2012/11/the-kudo-box.html ”There are many wrong ways to reward employees. A simple but effective approach is to install a “kudo box”, which enables people to give each other a small reward.” Tell a Story at the Daily Standup http://www.shino.de/2012/11/22/tell-a-story-at-the-daily-standup/ ”We should tell three stories during test reporting: a story about the product a story about testing a story about the process” Software testing training and gaming http://www.kohl.ca/2012/software-testing-training-and-gaming/ Discussion about using games to teach about software testing Censoring people for disagreeing with Context driven testing http://context-driven-testing.com/?p=55 Interesting drama in the test world TDD ignores the design – No it doesn’t http://www.renaissancesoftware.net/blog/archives/382 ”TDD ignores design; that is a frequently stated misconception. Many people get this idea from the code focus of TDD. TDD does not call for the creation of any non-executable design documentation. So the questioning developer gets the idea that there is no design. But I say, “yes there is”.” “But make no mistake about it, it you can’t get your code into a unit test, it’s not well designed.” TDD guides you to improved design http://www.renaissancesoftware.net/blog/archives/415 ”My premise from these first two articles is that TDD is radar for approaching code rot, as well as a guiding beacon to modular design.” ”When your code depends on some third party code, or code from some other subsystem or group that causes testing inconvenience, you have a choice to make. Should I create a test double for the problem dependency, or write an abstraction layer above the problem dependency. Given new code, the abstraction layer, or adapter, is usually a good choice.” Sikuli – Some first impressions
  • 14. http://www.mkltesthead.com/2012/11/sikuli-some-first-impressions.html ”Sikuli started as a project at MIT. It's now an open source tool that can be used with a number of different applications. It can be used with the web, it can be used with Flash apps, it can be used with compiled applications on a number of different platforms. Yeah, OK, that's all fine and good, but aren't there plenty of tools out there that already fit that space? Why use another one?” How to manage conflict in virtual teams http://blogs.hbr.org/cs/2012/11/how_to_manage_conflict_in_virt.html “Employee conflicts can be poisonous. We have all experienced the damage to productivity, crushed creativity, and squashed morale. As Kevin M. Campbell, Accenture's Group Chief Executive, Technology, notes, "All too often, I've seen that personal conflicts derail costly projects and important initiatives." Unresolved employee conflicts are bad enough in a traditional, physical workplace. They are all the more dangerous in a virtual environment, where people don't have the luxury of proximity to work their differences out face-to-face.” The Future of Software Security Testing http://blog.utest.com/the-future-of-software-security-testing/2012/11/ “In recent years it has been repeatedly proven that vehicle computer systems and wireless medical devices are susceptible to hacking. Though a malicious attack has never been reported, several research groups and security organizations have proven that security vulnerabilities exist that allow people to hack insulin pumps, pacemakers, implantable cardiac defibrillators (ICD) and wireless vehicle communication systems.” Another certification, another scam? http://thetesteye.com/blog/2012/11/another-certification_another-scam/ This is a continuous discussion in the testing world. Good to keep up-to-date about it if you are interested. Two ways to deal with Bias http://www.softwaretestpro.com/Item/5710/Two-Ways-to-Deal-with-Bias/ First part of the article is about creating a mind map of different areas of a function/application/feature and using that as a start for exploratory testing. Could be an interesting read. Where does all the time go http://www.developsense.com/blog/2012/10/where-does-all-that-time-go/ “So since you’re a tester, and since the door on the test lab says ‘Test Lab’, and your business card says ‘Testing’, they figure that’s all you do. The illusion is what Jerry Weinberg calls the Lumping Problem. All of those different activities—administrative compliance, setup, bug investigation and reporting, and test design and execution—are lumped into a single idea for them.” Why are there so many C++ testing frameworks http://googletesting.blogspot.se/#!/2012/10/why-are-there-so-many-c-testing.html ”So, I think a major reason that we have many C++ testing frameworks is that C++ is different in different environments, making it hard to write portable C++ code. Another reason is that some limitations of C++ make it impossible to implement certain features really well, and different people chose different ways to workaround the limitations. Finally, many C++ testing framework authors neglected extensibility and were happy just providing canned solutions,”
  • 15. The Less is Best Approach to Innovation http://blogs.hbr.org/cs/2012/10/the_less_is_best_approach_to_i.html ”In our world of excess everything, savvy innovators realize that less is actually best. They know that delivering a memorable and meaningful experience hinges on user engagement, which is best achieved through a subtractive approach. Anything excessive, confusing or wasteful is intelligently and cleverly removed, or never added in the first place.” The End of a Bug’s Life (Microsoft) http://experttesters.com/2012/10/16/the-end-of-a-bugs-life/ ”Some testers work on teams that reward employees based on the number of bugs they find. What effect does this have on testers? It encourages the finding of bugs, not the Closing of them. If anything, testers should be rewarded based on the number of bugs fixed and closed, which would at least encourage the resolution of quality bugs.” Test Managers and why you should not spend time trying to avoid blame http://testobsessed.com/2012/10/why-i-wont-go-back/ ”This *negative blame+ cycle flourishes only in organizations that perpetuate a culture of fear and blame. Fix the culture and you’ll fix the cycle.” The Problem of Organizational Change - People http://blogs.hbr.org/cs/2012/10/this_is_your_brain_on_organizational_change.html ”Although myriad factors are cited, the inability to engage people is the factor noted longest and most often. As organizational behavioral experts Kenneth Thompson and Fred Luthans noted almost 20 years ago, a person's reaction to organizational change "can be so excessive and immediate, that some researchers have suggested it may be easier to start a completely new organization than to try to change an existing one."” 100% Automation Pass Rate http://www.testingmentor.com/imtesty/2012/10/08/100-automation-pass-rates “Automated testing is now more pervasive throughout the industry as compared to the past. Automated tests running against daily builds are an essential element in an Agile lifecycle. Continuous integration (CI) requires continuous feedback from automated tests. So, let’s dispel some myths and explain why 100% pass rates are not only a good thing, but are and important goal to strive towards.” Why people don’t instantly buy into Agile http://secretsofconsulting.blogspot.se/2012/10/why-people-dont-instantly-buy-into.html “When "selling" their methods, Agile evangelists often stress the strength of Agile methods at removing, and even preventing, errors. I used to do this myself, but I always wondered how people could resist this sales pitch. I would plead, "Don't you want quality?" And, of course, they always said, "Yes, we want quality," but they didn't buy what I was selling. Eventually, I learned the reason, or at least one of the reasons. In today's blog, I want to help today's evangelists (coaches, team leaders, managers, or whomever) by sharing what I've learned about why Agile methods can be so difficult to sell.” The Rise of the Customer http://angryweasel.com/blog/?p=508
  • 16. ”I spoke recently on the topic of “Customer Focused Test Design” (synopsis: customers don’t really care how much functional testing testers do, or how many bugs they find. A test approach that favors scenarios, -ilities, and customer feedback is better; and a bunch of examples for emphasis/proof).” ”If you want to be a successful software company, you have to care about customers. In addition to keeping them happy once they have used your software or service, you need to respond to their needs and give them what they need” Making meetings productive http://blogs.msdn.com/b/jw_on_tech/archive/2012/10/02/making-meetings-productive.aspx “First, get the right attendees and be aggressive about it. Second, make coming and going kosher. Halfway through a meeting and you realize you can't contribute or don't need to be there? Leave. Third, vote with your feet by walking away from the more useless meetings. Just don't go to those meetings where the organizer holds court. Fourth, heed the warning signs. The first such sign is the meeting agenda. Big agendas mean lots of time switching topics. Fifth, follow up a scheduled meeting with scheduled work time. Finally, all this points to building an anti meeting culture within your organization” The Definition of Quality http://secretsofconsulting.blogspot.se/2012/09/agile-and-definition-of-quality.html ”In short, quality does not exist in a non-human vacuum, but every statement about quality is a statement about some person(s). That statement may be explicit or implicit. Most often, the "who" is implicit, and statements about quality sound like something Moses brought down from Mount Sinai on a stone tablet. That's why so many discussions of software quality are unproductive: It's my stone tablet versus your Golden Calf. When we encompass the relativity of quality, we have a tool to make those discussions more fruitful. Each time somebody asserts a definition of software quality, we simply ask, "Who is the person behind that statement about quality."” Premises of Rapid Software Testing http://www.developsense.com/blog/2012/09/premises-of-rapid-software-testing-part-1/ http://www.developsense.com/blog/2012/09/premises-of-rapid-software-testing-part-2/ ”These are the premises of the Rapid Software Testing methodology. Everything in the methodology derives in some way from this foundation. These premises derive from our experience, study, and discussions over a period of decades. They have been shaped by the influence of two thinkers above all: Cem Kaner and Jerry Weinberg, both of whom have worked as programmers, managers, social scientists, authors, teachers, and of course, testers.” Conversation with a Test Engineer at Google http://googletesting.blogspot.se/#!/2012/09/conversation-with-test-engineer.html ”Alan Faulkner is a Google Test Engineer working on DoubleClick Bid Manager, which enables advertising agencies and advertisers to bid on multiple ad exchanges. Bid Manager is the next generation of the Invite Media product, acquired by Google in 2010. Alan Faulkner has been focused on the migration component of Bid Manager, which transitions advertiser information from Invite Media to Bid Manager. He joined Google in August 2011, and works in the Kirkland, WA office.” Bugs spread disease http://testobsessed.com/2012/08/bugs-spread-disease/ ”There’s a lesson here about the real cost of tolerating bugs, of supporting practices that involve triaging defects. The cost of carrying a bug is far far greater than most people realize. That trivial little
  • 17. “cosmetic” issue will cost a little time to fix. If you have to argue about whether or not to fix it, then track it, then revisit it, then prioritize it, then argue again about whether to fix it or defer it yet again, you will spend hours upon hours on it. This is why I have no patience for the “bugs are inevitable; you can’t fix them all” attitude. Bugs kill. They do it slowly, painfully, but relentlessly.” I am an exploratory tester http://thetesteye.com/blog/2012/09/i-am-an-exploratory-tester/ ”I am curious about how the system works I look at details and the whole, and at many places I use many sources to learn what is important I am creative and see many testing possibilities I test in many different ways, and adjust to the situation I am good at finding important problems fast I am aware of models enabling and limiting my testing I can evaluate a system from many dimensions I can communicate my thinking and findings I am an exploratory tester” Our obsession with scale is failing us http://blogs.hbr.org/cs/2012/09/our_obsession_with_scale_is_fa.html In the industrial era and even the early web era, size was central to scale. The new reality is this: The ability to scale is no longer a direct function of size. Traditional Test Engineering … http://www.stickyminds.com/BetterSoftware/magazine.asp?fn=cifea&id=147 Turning Quality on its Head http://www.stickyminds.com/BetterSoftware/magazine.asp?fn=cifea&ac=492 ”In summary, the closer testing is related to real use the better equipped we will be to find the types of issues that are by definition the bugs that will most annoy users. Early cycle testing should thin the bug herd and eliminate the simple bugs and coding mistakes so that late cycle testing can focus on recall-class issues. Early cycle testing is like crowd control; no matter how diligent we are early in the cycle, later cycles of manual, dogfood, and beta testing are not skippable phases. Like it or not, we will benefit from putting more time and innovative thinking into late cycle testing.” Here is the original youtube video http://www.youtube.com/watch?v=cqwXUTjcabs How to get your bugs fixed (Microsoft) http://experttesters.com/2012/09/11/how-to-get-your-bugs-fixed/ ”To release good quality software, then, you not only have to be proficient at finding bugs, but you also have to be proficient at getting them fixed. Unfortunately some testers are good at the former, but not the latter.”
  • 18. Big Data’s Management Revolution http://blogs.hbr.org/cs/2012/09/big_datas_management_revolutio.html ”The evidence is clear: Data-driven decisions tend to be better decisions. In sector after sector, companies that embrace this fact will pull away from their rivals. We can't say that all the winners will be harnessing big data to transform decision making. But the data tell us that's the surest bet.” Recording My Testing Sessions http://enjoytesting.blogspot.se/2012/09/learning-from-rti-recording-my-testing.html ”Can you replicate the bug again for me? How many times have you answered the above question? It might be asked by testers, programmers, managers and anyone interested in the bug. Sometimes, even you will like to replicate the bug.” Ten Reasons to Fix Software Bugs Right Away http://blog.utest.com/10-reasons-to-fix-software-bugs-right-away/2012/09/ 1. Unfixed bugs camouflage other bugs 2. Unfixed bugs suggest quality isn’t important 3. Discussing unfixed bugs is a waste of time 4. Unfixed bugs lead to duplicate effort 5. Unfixed bugs lead to unreliable metrics 6. Unfixed bugs distract the entire team 7. Unfixed bugs hinder short-notice releases 8. Unfixed bugs lead to inaccurate estimates 9. Fixing familiar code is easier than unfamiliar code 10. Fixing a bug today costs less than tomorrow Getting 360 Reviews Right http://blogs.hbr.org/cs/2012/09/getting_360_degree_reviews_right.html ”But there is one thing we've personally seen that profoundly and consistently changes lives — what's generally referred to as the 360-degree feedback process. In the course of completing tens of thousands of these reviews as part of our strength-based leadership programs, we had an up-close view of many people who were teetering on the edge of job termination, and have seen them blossom into extremely valuable contributors. It's been one of the really gratifying parts of our work.” Testing 2.0 http://googletesting.blogspot.se/#!/2012/08/testing-20.html ”It’s amazing what has happened in the field of test in the last 20 years... a lot of “art” has turned into “science”. Computer scientists, engineers, and many other disciplines have worked on provable systems and calculus, pioneered model based testing, invented security fuzz testing, and even settled on a common pattern for unit tests called xunit.” Three Tips for Leaders about to Miss their Forecasts http://blogs.hbr.org/anthony/2012/08/three_tips_for_leaders_about_to_miss.html ”It's a scary moment when a company realizes that the goals it has set for a quarter or a year are simply not attainable. And yet in today's continually choppy environment, leaders are encountering that moment with increased frequency. And it is at this moment that leaders truly earn their pay, because many short-term actions can have devastating long-term consequences, particularly when it comes to innovation.”
  • 19. Interview Questions to ask your Future Employer http://trishkhoo.com/2012/09/interview-questions-to-ask-your-future-employer/ ”As interviewing is a two-way street, it’s always a good idea to ask your potential employer a few basic questions before accepting the gig. After all, you need to make sure that the company is as much a fit for you, as you are for them.” The Trojan Form of Change http://www.noop.nl/2012/09/the-trojan-form-of-change.html ”I believe real change in traditional management will not come from people telling managers how green the grass is on the other side of the fence. It will come from people who can inject an idea virus in a traditional organization, which is not destroyed by the current mental model, and which then transforms it from the inside, using the current mental model against itself." Intertwined Heuristics http://thetesteye.com/blog/2012/08/intertwined-sfdpot-crucspic-stmp/ Looking at two different models and seeing how they intertwine. Musings on Test Design http://angryweasel.com/blog/?p=502 ” The point is, and this is worth repeating so often, is that thinking of test automation and “human” testing as separate activities is one of the biggest sins I see in software testing today. It’s not only ineffective, but I think this sort of thinking is a detriment to the advancement of software testing. In my world, there are no such things as automated testing, exploratory testing, manual testing, etc. There is only testing. ” Testers caught sleeping on the job http://experttesters.com/2012/08/13/testers-caught-sleeping-on-the-job/ ”At Microsoft, we submit every code change to a peer-review before it’s checked in. I’ve performed hundreds (although it feels like millions) of code reviews in the six years I’ve been in Test. One of my biggest red flags during a code review is the Sleep statement.” A tool for mapping your goals http://blogs.hbr.org/cs/2012/08/a_tool_for_mapping_your_goals.html ”With multiple options on the table, it's hard to decide what's best for you and your future. But by assessing your career objectives and defining which are most important, you can reach a career decision with confidence. An easy but structured exercise like the career matrix can give you the answers you need all in one place.” Introducing the Software Testing Ice Cream Cone http://watirmelon.com/2012/01/31/introducing-the-software-testing-ice-cream-cone/
  • 20. Are you sure you are not a bad boss? http://blogs.hbr.org/cs/2012/08/are_you_sure_youre_not_a_bad_b.html “When we analyzed the behavior of 30,000 managers, as seen through the eyes of some 300,000 of their peers, direct reports, and bosses on 360-degree evaluations, we found that the sins of the bad boss are far more often those of omission, not commission. That is, bad bosses are defined not so much by any appalling things they do as by certain critical things they don't do.” Orchestrating Test Automation http://angryweasel.com/blog/?p=496 ”Designing good tests is one of the hardest tasks in software development. That’s worth saying again. Designing good tests is one of the hardest tasks in software development.” ”I fear that testers are worrying too much about the effort and skills required to automate tests – and not enough about what it takes to design reliable, portable, accurate, and trustworthy tests that actually matter.” Yet another Software Testing Pyramid http://watirmelon.com/2011/06/10/yet-another-software-testing-pyramid/ We Have Met the Enemy and He Is PowerPoint http://www.nytimes.com/2010/04/27/world/27powerpoint.html?_r=3&src=me&ref=general “It’s dangerous because it can create the illusion of understanding and the illusion of control,” General McMaster said in a telephone interview afterward. “Some problems in the world are not bullet-izable.”
  • 21. Quality is a Four Letter Word http://testastic.wordpress.com/2012/06/23/quality-is-a-4-letter-word/ ”In my experience, metrics of this sort serve to do one thing and one thing only. They seek to modify behavior of the test team. This is due to the Hawthorne Effect. Peoples’ behavior change in accordance to how they are being measured. As an experienced manager, I know the ones I chose above, if used “right”, will make sure more and more tests are being produced, but they tell me nothing of the quality of the code or product.” Initial Thoughts on Group Testing http://thetesteye.com/blog/2012/07/initial-thoughts-on-group-testing/ ”I want to open for a discussion on pair testing or at least widen the concept. By saying pair, we say two people, but why should we limit us to that? Depending what your objective is a different number might be more applicable?” A List of Distorted Thinking http://marlenacompton.com/?p=3325 “Being human means that if you have a bad lunch or some cloud system goes down which keeps you from your testing, the physical reaction of frustration leaves you very open to distorted thinking. In the case of testing, some of this distorted thinking seems useful for uncovering test ideas, and yet we also need to recognize when it’s time to stop catastrophizing and time to start collaborating with our teammates.“ Actually there are best practices http://www.shino.de/2012/07/29/actually-there-are-best-practices/ Discussion about what Best Practice means. The Testing Quadrants – We got it wrong http://www.shino.de/2012/07/30/the-testing-quadrants-we-got-it-wrong/ ”With the new labels the quadrants are a bit different than the ones that Brian Marick originally wrote down. On the left-hand side the confirmatory tests that are technology-facing are of course the unit tests and those class-level integration tests that developers write. These help drive the development of the project forward. In the second quadrant now are the expectations of the business. Usually I try to express most of them in the form of executable specifications, that I can automate. And I can also imagine other business-facing expectations that I cannot automate that probably fall in here, like reliability of the software product. In the third quadrants on the top right are tests that help to investigate risks concerning the external quality of the product. For me Exploratory Tests fall in this category, but also usability concerns, and end-to-end functionality. The fourth quadrant then probably consists of internal qualities for the code like changeability (static code analysis comes to mind), maintainability (how high is your CRAP metric these days?), and design flaws like code smells.” Creating a Test Lab for Mobile Apps Testing http://www.kohl.ca/2012/creating-a-test-lab-for-mobile-apps-testing-part-1/ http://www.kohl.ca/2012/creating-a-test-lab-for-mobile-apps-testing-part-2/ http://www.kohl.ca/2012/creating-a-test-lab-for-mobile-apps-testing-part-3/
  • 22. ”If we have a testing burden when we develop mobile apps, what sort of considerations should we take into account when we are setting up test labs, when using real devices? Here are some pointers from my own experience.” Three Keys to Mobile Application Design http://www.kohl.ca/2012/three-keys-to-mobile-application-design-part-1/ http://www.kohl.ca/2012/three-keys-to-mobile-application-design-part-2/ “If you are new to designing mobile applications, that often means you will look at web apps and PC apps first, because that is what you are most familiar with. Then you pull out your smartphone, and look at the apps you like and use the most there. That’s exactly what I did when I started out, and while I felt like I had adjusted to a new paradigm (smaller screens, less powerful devices, different network technology, etc.) I felt like something was missing. I couldn’t quite put my finger on what was different, but the apps I use on my mobile devices, and the apps I helped create demonstrated it to me. We are going through a major shift in technology, and the software we use is moving toward a combination of mobility, social connections, and gaming and entertainment.” FEW HICCUPPS Heuristics http://www.developsense.com/blog/2012/07/few-hiccupps/ Several years ago, I wrote an article for Better Software Magazine called Testing Without a Map. The article was about identifying and applying oracles, and it listed several dimensions of consistency by which we might find or describe problems in the product. However, it’s been a long time since the article was published, and since then, James Bach and I have observed testers using other oracle principles, both to anticipate problems and to describe the problems they’ve found. Keep experts on tap, not on top http://blogs.hbr.org/cs/2012/07/keep_experts_on_tap_not_on_top.html Among the many qualities that distinguish successful leaders from millions of less-successful executives in the world is an awareness of the limits of their knowledge. They know what they know, they appreciate what they don't know, and they have a healthy respect for what they don't know they don't know. In short, they have great meta-knowledge. In my experience, experts are among the least successful predictors in times of massive uncertainty. Valve Handbook for New Employees http://www.valvesoftware.com/company/Valve_Handbook_LowRes.pdf Tracking Testing on the Scrum Taskboard http://www.shino.de/2012/07/04/tracking-testing-on-the-scrum-taskboard/ ”How do you treat bugs on the taskboard that are found during testing? Create a new test for each bug, and put the test task back in ToDo? Or create a bug, and a bug-follow-up testing task? ” Two problems with Context Driven Testing http://www.shino.de/2012/07/04/two-problems-with-context-driven-testing/ The most interesting thing about this articles is actually the comments by Cem Kaner: “As I see it, my responsibility (as a tester) is to serve the project and its stakeholders, in ways that advance their goals for the project, and if I can’t focus on that and do that well, I shouldn’t be on that project.” Test Estimation Tips
  • 23. http://experttesters.com/2012/07/09/test-estimation-tips-but-no-tricks/ ”Estimating the time it takes to test a feature is an important, yet often overlooked, skill. If your estimate is too low, it can affect your product’s quality and release date. If your estimate is too high, it can lead to an inefficient use of resources. The topic of project estimation is complex enough topic to fill entire books. But here are a few simple tips that can quickly improve your estimation skills, not to mention your work/life balance and general happiness.” The Growing Queue at the IT Morgue http://blogs.msdn.com/b/jw_on_tech/archive/2012/06/29/the-growing-queue-at-the-it-morgue.aspx ”For one, hardware and user-experience companies like Apple, who make their money selling products and apps, are more valuable than web companies like Google who make their money advertising web content. This value gap is only going to get wider. As the web recedes into the background, the opportunity shifts from making money on ads to providing actual value to users monetized through direct sales and subscriptions. Devices and data centers are the future and it is a very compelling future.” Estimates are often helpful, Estimates are often not http://www.estherderby.com/2012/03/estimating-is-often-helpful-estimates-are-often-not.html ”Estimating helps when the process of estimating builds shared understanding among the people who want the work done and the people doing the work.. Collaborative estimating gives the best results. Diverse experience yields a broader range of perspectives and questions. Questions and perspectives build understanding of the what, why, and who related to the request. That’s helpful. Group estimating reveals differences in knowledge and understanding. Finding those gaps early is helpful. Group estimating surfaces assumptions. When we are aware of our assumptions, we can verify–or debunk– them.” The Hallmark of a Great Test Team http://experttesters.com/2012/06/18/ostrich-a-la-mode/ “One hallmark of a great test team is how they behave in the final days before a release. I’m not referring to working late to automate the last few tests, validate bug fixes, and run a final test pass. These long hours are an expected part of the job, and when you think about it, these tasks are really just bookkeeping. I’m talking about something that actually affects the product: does your test team still try to find bugs just before a release?” The elements of Improvement (Three pre-requisites for improvement) http://www.estherderby.com/2012/05/the-elements-of-improvement.html Information. People need information about the context and how their work fits into the big picture. They need information from the work so they can self-correct. Without this information, systematic improvement is impossible. A desire to improve. Most people want to do their best and learn to do better–until that impulse is squashed. One-sided evaluations, organizational hurdles, relentless pressure strangle the desire to improve. Time to reflect and learn. People need time to design and implement new processes, and practice new skills. Relentless deadline pressure makes this impossible. People under pressure are less likely to try something new, or think clearly about anything. What do middle managers do in Agile? http://www.estherderby.com/2012/04/what-do-middle-managers-do.html
  • 24. ”Organizations moving to agile still need management, and often still need people in management roles, especially in large complex organizations. In traditional hierarchies, middle managers look up the hierarchy for direction, and focus down the hierarchy to accomplish cascading goals. When teams pull work from queues and self-organize to meet goals, the real opportunity for middle managers is to look across the organization to improve the system and develop people and teams.” Test Plans & Test Cases in Agile/Scrum http://www.rbcs-us.com/blog/2012/06/08/test-plans-and-test-cases-in-agilescrum/ ”First, yes, the “where’s the testing in Agile?” question is a good one. Many of the people involved in defining Agile methodologies seemed very focused on automated unit testing, and perhaps missed the fact that unit testing by itself is not enough. According to Capers Jones’ studies, even the best unit testing removes only 50% of defects, so testing needs to be added to the methodology.” Personal Kanban Tool (Free for personal use or for small teams, costs money for more users) http://leankitkanban.com/ Can TDD Help Reduce Integration Time? http://www.renaissancesoftware.net/blog/archives/364 ”It has been known for decades that the sooner you find a problem the easier it is to fix. TDD takes that idea and applies it to the minute to minute activity of programming. Not that TDD is only about defect prevention, but each defect prevented can potentially save considerable time in the future, including time spent in integration.” Manual Test is Unsustainable http://www.renaissancesoftware.net/blog/archives/206 ”Creating automated tests can be very difficult, especially when the code has gotten long in the tooth and was not created with automated tests to begin with. Many product development teams don’t invest in automated tests. They think they cannot afford them. They think their product is different and can’t be manually tested. This thinking is flawed.” Management 3.0 Is a Way of Work http://www.noop.nl/2012/06/management-30-is-a-way-of-work.html ”My core belief is that organizations are complex adaptive systems, and that managers must care for the system, instead of bossing people around.” Ethics of Testing (James Bach) http://www.satisfice.com/blog/archives/771 ”I have an ethical code. Actually, several! The Association for Software Testing adopted the ACM Code of Ethics, verbatim. It’s okay. I would have preferred the simpler IEEE Code, personally. I also almost completely follow Jerry Weinberg’s code. I take comfort from all of them, plus I have my own.” Microsoft vs. Google Part 1: Innovation http://blogs.msdn.com/b/jw_on_tech/archive/2012/05/25/google-20-time-vs-the-microsoft-garage.aspx Their grassroots efforts at innovation clearly show they believe (correctly) that creativity isn't governed by a person’s title or the org in which they sit. Creative people can literally be anyone and are often those without advanced degrees (Gates, Jobs, Zuckerberg …) and without an entrepreneurial track record (many top innovators have been one hit wonders). However, they take a very different approach to encouraging the masses to innovate. The bottom line: with 20% time
  • 25. Google lowered the bar for innovators and thereby watered down the value of the results; with its Garage, Microsoft raised the bar and created a more sustainable model. Sleepy Automated Tests http://www.testingmentor.com/imtesty/2012/06/11/sleepy-automated-tests/ “One of the most common problems with automated tests is synchronizing the automated test and the application or service under test. Automated tests sometimes race through their instruction set faster than the application or service can respond leading to false positive (test reports a failure but there is no bug). Often times testers use a Band-Aid approach to solve the problem by sprinkling Sleep() methods throughout their test code in an attempt to synchronize the test code and the application or service under test. Unfortunately, these Sleep() methods may negatively impact the performance of an automated test and artificially increase the time of the automated test pass.” The Goal and the Path http://angryweasel.com/blog/?p=461 ”I’ve seen many people with “visions” fail to succeed because they spent their time looking for shortcuts through the system rather than understanding that navigating the system is the key to achieving a vision (conversely, I’ve seen many others fail because they focused on navigating the system rather than their initial goals).” Management 2.0 is the right thing done wrong http://www.noop.nl/2012/06/management-20-is-the-right-thing-done-wrong.html ”There’s also nothing wrong with the idea behind balanced scorecards. The problem with measurements is that one metric will lead to sub-optimization, and therefore you need multiple metrics to have a holistic view of the organization. It is useful to have multiple perspectives on complex problems. Unfortunately… managers see the organization as a collection of parts. They try to optimize departments. But in a complex system the performance is in the relationships between the parts. And thus you need metrics that cover perspectives of multiple stakeholders, or “measure from-the- outside-in”.” Proving the Embedded Software Test Team http://www.softwaretestpro.com/Item/5548/%27Proving%27-The-Embedded-Software-Test-Team/ ”I am afraid that I do not have a lot of quantitative data, as most software engineering metrics lack construct validity. (Or, in different words, "All bugs are not created equal.") Three arguments I have for embedded teams: 1) Track the amount of time testers spent blocked on different clients over a period of years, I see a correlation between tester-is-expensive-paperweight waiting for a build and external test teams. Of course, there are too many independent variables to track here; the improved teams didn't just embed the testers, they also generally moved toward one-piece flow, pair programming, etc. 2) Track the number of handoffs that occur, from dev-to-test-to-dev. When people sit next to each other, it is easier to show the dev your test methods, so the programm can 'poke test' the app before handing it back - not just fix the first blocking bug you found as a tester. Also, allowing mentions-in- passing and desk pairing increses the chance the bug will get fixed-fixed on the first go round. 3) Track the costs of hand-offs. If testers have to file a bug report that needs to get triaged before it
  • 26. gets fixed, then costs increase over a conversation and a fix. The most project management layers on top of this, the more the fixes cost.” Using Project Artefacts to their full potential (The same thoughts are applicable to many test artefacts as well) http://thecaymansays.blogspot.se/#!/2012/05/hello-world-today-i-realized-its-been.html “Problem: People in projects are very seldom using the basic artefacts to it's full potential. Which artefacts? Time plans. Resource committments. Budget plans. Processes and delivery plans.” Visualizing Stakeholder relationships http://thecaymansays.blogspot.se/#!/2012/05/thursday-evening.html ”Just a thought about stakeholder maps. If you're into 6 Sigma, you know what I mean. Quite possibly you know about them anyway. Four quadrants, Interest or Support on the x-axis, power or influence on the y-axis. Map people according their interest in the project and their power in the organization. Anyway, when I did my 6 Sigma Green Belt training, we were advised not to show the stakeholder map to the people actually listed there. What if we actually did the opposite and took the stakeholder map to all concerned parties and showed it to them and asked them to comment it?” A sense of ownership http://www.noop.nl/2012/05/a-sense-of-ownership.html ”A friend of mine recently told me that his off-shore team was producing terrible results, and slowly too. When he traveled there to discuss the quality and productivity problems, he found out that the same people had their own little startup on the side, working on it in their spare time. And the product they were making as a startup was amazing. While the product they were making as off- shore employees was crap. I think these people had an ownership problem.” We Cannot Choose Between Management And Leadership http://www.jrothman.com/blog/mpd/2012/05/we-cannot-choose-between-management-and- leadership.html “I subscribe to a number of services that look for pithy quotes from Big Names, authors, and other people who are looking for publicity. I saw one about moving from manager to leader. Ok, so these are writers or reporters, and they may not know. Choosing to be a manager without being a leader is like choosing to drive across the country without a map.” Metrics & Ethics http://context-driven-testing.com/?p=49 They (management) understood that the metrics were imperfect. But they felt that they needed ways to summarize what the organization knew about projects. They felt they needed ways to compare progress, costs, priorities, and risks. They felt they needed ways to organize the information so that they could compare several projects or groups at the same time. And they felt they needed to compare what was happening now to what had happened in the past. Hung then made three points: 1. These are perfectly legitimate management goals. 2. Quantification (metrics) is probably necessary to achieve these goals. 3. The fact that there is no collection of metrics that will do this perfectly (or even terribly well) doesn’t eliminate the need. Without a better alternative, managers will do the best they can with what they’ve got. .
  • 27. It is important (I think very important) to pay attention to the validity of our metrics. It is important to improve them, to find ways to mitigate the risks of using them, and to advise our clients about the characteristics and risks of the data/statistics we supply to them. More adventures in magic automation numbers http://adam.goucher.ca/?p=1767 ‘When will our *functional+ automation remove the need for our extensive manual regression testing?’. This is of course a trap. There is no way to determine this number with any degree of certainty. But what if you really needed to provide an argument around the inability to predict this? Wherever You Go: Testing Mobile Applications http://www.techwell.com/articles/weekly/wherever-you-go-testing-mobile-applications-part-1 http://www.techwell.com/articles/weekly/wherever-you-go-testing-mobile-applications-part-2 Joey McAllister: What are some particular concerns you’ve encountered with popular mobile products like games, social applications, and mapping software? Jonathan Kohl: A couple of the big things that are difficult are security and privacy. The devices themselves aren’t that secure with their network connections, so it’s easy for people to snoop in, crack your accounts, and find out personal information. But, privacy is a huge issue. Some of the big social applications get in trouble in Canada because they violate our privacy laws and they have to clean things up. Two of the most dangerous terms in Agile; Done and Whole Team http://adam.goucher.ca/?p=1771 I have come to realize in the last two weeks that there are two words in the Agile lexicon that really bug me. Not because they are bad per se, but that they are so commonly misused and abused. Not to mention being insanely context sensitive. The Secrets of Faking a Test Project http://www.kohl.ca/2012/the-secrets-of-faking-a-test-project/ This presentation is satirical – we challenge people to think about how they would approach a project where the goal is to release bad software, but you make it look as if you really tried to test it well. It didn’t take much effort on our part, we just looked to typical, all too common practices that are often the staple of how people approach testing projects, and presented them from a different angle. Why 100% Test Pass Rates are bad http://experttesters.com/2012/05/17/why-100-test-pass-rates-are-bad/ When you change your mindset and write tests to prove the product doesn’t work, rather than to prove that it does work, you’ll be surprised how many more bugs you expose. This simple change will make you a better tester, and your products will be better for it. Testing the Limits with Zynga’s Galina Kramer (Localization Testing) http://blog.utest.com/testing-the-limits-with-zyngas-galina-kramer/2012/05/ uTest: Aside from being challenging, localization is arguably one of the most misunderstood testing types. We don’t expect you to disclose any specific bugs, but what are some of the types of issues
  • 28. that are uncovered during the course of a localization test? Misspellings? Poor translations? Others? And while we’re at it, how do you define localization testing? GK: Localization testing covers all of the above and more. We definitely look for misspellings, poor translations, truncation issues… We need to make our games equally enjoyable in every language. This means keeping in mind culture differences, religious beliefs, language intricacies – all of it. Testing with surrogate code points http://www.testingmentor.com/imtesty/2012/05/21/testing-with-surrogate-code-points/ Surrogate pairs are often problematic in string parsing algorithms. Unlike “typical” 16-bit Unicode characters in the base multilingual plane (BMP or Plane 0) surrogate pairs are composed of 2 16-bit Unicode code points that are mapped to represent a single character (glyph). (See definition D75 Section 3.8, Surrogates in The Unicode Standard Version 6.1) Surrogate code points typically cause problems because many string parsing algorithms assume 1 character/glyph is 1 code point value which can lead to character (data) corruption or string buffer miscounts (which can sometime lead to buffer overflow errors). As an example of string buffer miscounting let’s take a look at Twitter. Seven Ways to Find Software Defects Before They Hit Production http://www.informit.com/articles/article.aspx?p=1888112 This article provides a quick list of test ideas in seven techniques, along with hints of where to go for more: 1. Quick Attacks 2. Equivalence and Boundary Conditions 3. Common Failure Modes 4. State-Transition Diagrams 5. Use Cases and Soap Opera Tests 6. Code-Based Coverage Models 7. Regression and High-Volume Test Techniques Quality Assurance is a Process, not a Department http://www.forbes.com/sites/goodmenproject/2012/04/18/the-quality-chronicles-the-bug-that-killed- an-ipo/ “Of course every great development company will have a final step in the process called Quality Control or Quality Assurance, but it is my sense that the QA formal group is there to be the standard- bearer for Quality and rally the company around it, putting a final go or no-go procedure in place before the world gets its hands on a product, but not accepting proxy status for an otherwise poor process. To ignore or be ignorant of a showstopper in one’s own product is a reflection of a process that needs to be re-engineered. When you’re working with world-class engineers, it is much easier and far more fruitful to make sure the process is engineered correctly before the products go through it. The material cost of discovering a bug early in development is a tiny fraction of what it can cost you in the hands of the public. Give your engineers a voice and they will save you every time.” The Embedded Tester http://managetotest.com/2012/04/12/the-embedded-tester/ ”I hear it a lot at conferences, work meetings, interviews, discussions with managers, etc., the idea of “embedded testers.” The utterances are along the lines of, “Our testers are embedded on development teams!” or “How do we embed our testers with our developers?” It unfailingly puts me
  • 29. in the mind of a reporter from say, CNN wearing an ill-fitting helmet and flak jacket*, standing there as well-drilled troops go about their business. In Agile, testers aren’t “embedded” on teams any more than programmers are, or analysts, or any other role that is needed on the team. To say that they are suggests that this is an option, or a particular strategy you might employ to help with Agile development. It’s not! It’s an essential part of it!” Eliminate boring testing: Automating visual comparison http://trishkhoo.com/2012/04/eliminate-boring-testing-automating-visual-comparison/ ”It was a pretty straightforward process to set up an automated script that would import a campaign and show the preview screen. Then I just needed a way to capture the previews in each environment as a screenshot for easy comparison. Visually comparing two screenshots was still pretty annoying and time-consuming. So I added a step to automatically compare the two images pixel by pixel, and tell me if they’re identical. If they’re not identical, then I want a new screenshot showing me how they’re different.” The Secrets of why Testers are valued and proud at Microsoft http://experttesters.com/2012/04/17/secrets-of-sdet-success/ “The short answer is, hire great people, give them challenging work, and give them a vision for growth.” Testing Dojo Example http://www.shino.de/2012/04/02/testing-dojos-in-kyiv-ukraine/ ”The dojo started with the introduction of the rules and our mission statement. The attendees were going to deal with touring heuristics FCC CUTS VIDS and pass as many of the tours as possible.” Why internet search as we know it today will disappear http://blogs.msdn.com/b/jw_on_tech/archive/2012/03/15/why-i-hate-search.aspx http://blogs.msdn.com/b/jw_on_tech/archive/2012/04/13/now-you-see-it-in-20-years-you-won-t.aspx ”The problem with Internet search is that being stupid about it is profitable. The more ugly blue links you serve up, the more time users have to click on ads. Serve up bad results and the user must search again and this doubles the number of sponsored links you get paid for. Why be part of the solution when being part of the problem pays so damn well?” Misusing Software Process Metrics as People Metrics http://www.rbcs-us.com/blog/2012/03/26/misusing-software-process-metrics-as-people-metrics/ “I wish I could say that I never saw organizations make this kind of mistake with process metrics (i.e., mistaking a software process metric for a people metric), but unfortunately it is all too common.” Failing Fast http://managetotest.com/2012/04/16/failing-fast/ “MacCready took a different approach. He knew that no matter how good his designs, the likely result would be failure, just like all the others before him. As a result, he expected failure and built his prototypes with that in mind. He use common and cheap materials that were easy to work with and easy to repair and replace. As a result he could test new designs in a matter of weeks rather than years.” Indicators, Testing & Wine Tasting
  • 30. http://testers-headache.blogspot.se/2012/04/indicators-testing-wine-tasting.html ”Essentially, it was about how to judge feedback from a continuous integration system next to the other information that the testers in the team were producing. The reason I'd started asking questions was that with the myriad of information available there was a risk (as I saw it) to ignore some information - or cherry-pick the information to use...” Everything You Wanted to Know About SCRUM, But Were Afraid to Ask http://blog.utest.com/video-everything-you-wanted-to-know-about-scrum-but-were-afraid-to- ask/2012/04/ Want to learn about SCRUM, but don’t have 10 minutes of spare time? Then checkout this 9-minute video on the basics of this popular methodology. A nice introduction to a very important subject. Enjoy! Test doesn’t understand the customer http://testastic.wordpress.com/2012/03/31/knowurcustomer/ ”On the first day of our mentorship, I asked his opinion around something I had casually observed at Microsoft. What I did not expect was how fundamentally his answer would shake my world. Me: “…In addition, given your experience, there’s another pattern I’ve observed that I’d love your opinion on. I’ve noticed that around 15 out of 20 folks in your role (Product Business Manager) got there via Program Management, about 4 via development, and around 1 out of 20 via test.” Mr. R: “I don’t know if that’s totally accurate, but seems about right.” Me: “So why do you think test so rarely gets the role?” Mr. R: “Oh, That’s easy! It’s because Test doesn’t understand the customer.” ” What is “leadership” anyway? http://www.softwaretestpro.com/Item/5481/What-is-%27leadership%27-anyway/ “It's easy to see the absence of leadership - but how do you define it's presence?” “We've replaced leadership with management, and we suffer from it. This replacement pushes us, to a great extent, to many of the maladies you've seen on this blog: The desire to institutionalize process, to have easy answers, to manage to a template or a spreadsheet or process instead of getting out there, understanding the problem, and, you know, fixing it!” Why I joined Microsoft by James Whittaker http://blogs.msdn.com/b/jw_on_tech/archive/2012/03/14/why-i-joined-microsoft.aspx “I suppose that pointing out that the Google to Microsoft transition is not particularly rare won't suffice as an explanation so what follows is a more long winded account. For those uninterested in the finer points, here's the short version: I think much of what is happening in the mobile and web space is broken and is trending toward becoming more broken. Users have their privacy under assault and are losing control of their identity and personal data. Independent developers are facing walled gardens around the data and services necessary to move the web forward. Solving this problem is going to require a diverse set of intellectual property, technical and information assets and an inclusive attitude toward software developers. In my opinion, Microsoft is among the best positioned companies to lead such an effort.” We're Not So Different, You And I http://www.noop.nl/2012/03/were-not-so-different-you-and-i.html ”No matter where I go, people have the same problems with management & leadership, the same problems with self-organizing teams, and the same problems with organizations & departments.” “In my experience, cultural differences between countries pale in comparison to the cultural differences between industries.”
  • 31. 97% http://blogs.msdn.com/b/anitag/archive/2012/04/03/10290584.aspx ”Our test cases shouldn't be written to prove there are bugs in our code. They should be written to prove there are no bugs in our code. So having bugs associated with 97% of our test cases in one of our projects means there are way too many bugs in our code and we need to push quality upstream. ” Look to the Data http://experttesters.com/2012/03/20/look-to-the-data/ “I was recently asked to investigate my team’s automated daily test cases. It was taking more than 24 hours to execute our “daily” test run. My job was to find why the tests were taking so long to complete, and speed them up when possible.” Exploring Test Roles http://angryweasel.com/blog/?p=444 ”Roles that testers play on teams vary. They vary a lot. You can’t compare them. That’s ok, and (IMO) part of the growth of the role. I find it disturbing and detrimental when testers not only assume that their version of testing is “the way”, or that some roles (or people in those roles) do not qualify as testing.” Every Test Failure Means Something http://angryweasel.com/blog/?p=432 ”The point of this story is simple. Every test failure means something. The failure may mean a product failure. It may mean you have flaky tests. When you start to assume flaky tests or environments, you’re heading into the land of broken windows and product failures you could have found earlier (actually, you probably did – you just ignored them).” “Relying on test automation for any part of your testing is pointless if you don’t care about the results and look at failed tests every time they fail.” The Robots are Taking Over http://angryweasel.com/blog/?p=437 “Normally, I don’t blog about news stories, but on the way to work this morning I heard an interesting discussion on a local talk radio station. It turns out that there are people up in arms over the purchase because using robots in the warehouse puts people out of jobs. I laughed out loud in my car as I realized that these people calling into the radio station were having the same discussion many people have about test automation.” Testing the Limits with Gerald Weinberg http://blog.utest.com/testing-the-limits-with-gerald-weinberg/2012/03/ Bug Title Crash Course http://thetesteye.com/blog/2012/03/bug-title-crash-course/ * As short as possible, but no shorter than that. * Start the sentence with the most important, to capture the reader’s interest. * Don’t overdo “externalization” to capture interest, rather describe the dry facts, and let the readers draw conclusions
  • 32. * If it’s difficult, try many times, or write the title after everything else is done (you might find the right words on the way.) * Include a brief description of what happens. * Include where the problem happens. * Describe observations rather than (presumed) facts. * Use a fair description, don’t exaggerate or understate the consequence of the problem The Linear Filter Trap http://www.developsense.com/blog/2012/03/i-might-be-wrong/ ”People like to solve problems that they can see - which means we have a tendency to "fix" the problem we see in front of us. This occurs more often if the problem appears to have a "straightforward" fix -> I think of this as a form of cognitive ease in action. Digging for root causes is a challenging activity, and we sometimes want to believe that the cause we identify is good enough to fix.” Delegation does not mean Negation http://www.noop.nl/2012/03/delegation-does-not-mean-negation.html ”With delegation of work you do not delegate your accountability. Instead, you share it.” Context Driven Testing http://scott-barber.blogspot.com/2012/03/context-driven-testing-crossroads.html http://discoveredtester.blogspot.com/2012/03/is-context-driven-school-of-testing.html http://scott-barber.blogspot.com/2012/03/with-context-driven-school-closed-whats.html http://www.mkltesthead.com/2012/02/in-support-of-context-driven-principles.html http://www.developsense.com/blog/2012/03/i-might-be-wrong/ http://xndev.blogspot.com/2012/03/lets-call-calling-off-off.html Sleeping with the enemy - Gojko Adzic describes why independent testing should be a thing of the past. http://oredev.org/2011/sessions/sleeping-with-the-enemy Traditional software delivery models are based on a lack of trust. Because the business doesn't trust developers, testers are asked to provide independent validation. Because developers don't trust testers, everyone wastes a lot of time arguing about whether a problem is in the code or in the tests. And testers are taught not to trust anyone! All of this distrust even though we share the same end- goal—delivering a product that satisfies our customers. Gojko Adzic describes why independent testing should be a thing of the past. Focusing Testing on Business Needs http://oredev.org/2011/sessions/focusing-testing-on-business-needs Testers often forget that they are service providers whose role is to provide critical information to the project’s stakeholders. Testing must focus on business needs to add the most value and gain respect. Artful Testing http://oredev.org/2011/sessions/artful-testing In this presentation I will investigate what happens when we infuse testing with aesthetics. Can the fine arts in any way support or complement our testing efforts? How I wish users knew how I help them through context driven testing
  • 33. http://oredev.org/2011/sessions/how-i-wish-users-knew-how-i-help-them-through-context-driven- testing This talk will be about the journey of being a context driven tester and the opposition I faced and how I paddled through it to win in most of my assignments. Context-driven Testing is not a religion http://context-driven-testing.com/?p=23 Over the past few years, several people have gained visibility in the testing community who express ideas and values that sound context-driven to me. Some call themselves context-driven, some don’t. My impression is that some are being told they are not welcome. Others are uncomfortable with a perceived orthodoxy. They like the approach but not the school. They like the ideas, but not the politics. Contexts Differ http://context-driven-testing.com/?p=38 We don’t manage the project managers. We don’t decide what information they have to give to the people they report to. Context-driven testing at a crossroads (James Bach and Cem Kaner part ways) http://www.satisfice.com/blog/archives/724 ”Cem Kaner, who controls www.context-driven-testing.com, has announced an interesting change in his view of the Context-Driven School. He says he prefers to think of it in terms of the Context-Driven approach, not a school of thought. This is a significant change from his original view, which was that CDT is a different paradigm.” Lessons Learned from Context-driven testing (Follow up to the above article – with comments from Cem Kaner which are interesting) http://www.shino.de/2012/02/29/lessons-learned-from-context-driven-testing/ There has been some fluff and rumor around context-driven testing yesterday. Some folks even talked about the death of context-driven testing. Most of it was issued by the about page from Cem Kaner. If you haven’t read it yet, go ahead, read it now, I will wait here. The Hindsight Trap http://testers-headache.blogspot.com/2012/03/hindsight-trap.html Root cause analysis (RCA) activities are an important part of many retrospectives - whether as part of a team activity, analysis of test reports, analysis of fault (bug) reports or as standalone activities to investigate organizational and process problems. The most common form of this trap in testing circles (where documentation is involved) usually crops up as a question, "why didn't you/someone spot this problem earlier (as it's specified in document X)", or statement, "this requirement should been tested because it's specified in document X". 60% of Users Expect Mobile sites to load in 3 Seconds or Less (Importance of Load testing for mobile web) http://blog.utest.com/60-of-users-expect-mobile-sites-to-load-in-3-seconds-or-less/2012/03/ That’s right, if your mobile site doesn’t load within THREE seconds 60% of visitors will abandon it. And, seeing how we’re not a very patient society anymore, 57% of mobile users will only give your site an extra two seconds (bring the grand total up to five) to load before aborting the mission.
  • 34. Test Reporting by Michael Bolton http://www.developsense.com/blog/2012/02/why-pass-vs-fail-rates-are-unethical/ Calculating a ratio of passing tests to failing tests is a relatively easy task. If it is used as a means of estimating the state of a development project, though, the ratio is invalid, irrelevant, and misleading. At best, if everyone ignores it entirely, it’s simply playing with numbers. Otherwise, producing a pass/fail ratio is irresponsible, unethical, and unprofessional. http://www.developsense.com/blog/2012/02/braiding-the-stories/ Testing is like investigative journalism, researching and delivering stories to people. The newspaper business knows how to direct attention efficiently to the stories in which we’re interested, such that we get the level of detail that we seek. http://www.developsense.com/blog/2012/02/delivering-the-news-test-reporting-part-3/ As testers (or, heaven help us, quality assurance) people, we tend to be chartered to look for problems, and to investigate them in ways that are most helpful to programmers, managers, and our other clients. A failing test on its own tells us little, and a failing check even less; a failing test is only an allegation of a problem. Investigation and study of a failing test is likely to inform us of something more useful: whether someone will perceive a problem that threatens the value of the product. What Metrics do you Use in Agile? http://testobsessed.com/blog/2012/02/23/question-from-the-mailbox-what-metrics-do-you-use-in- agile/ This is a great question. Too many organizations transitioning to Agile don’t ask it. They continue using the same metrics they were using for traditional projects even though the process is fundamentally different. The problem is that traditional measures don’t work in Agile. Sometimes they’re actively harmful. First, I only use metrics to get a 50,000 foot view on what’s going on. Second, I do not use metrics to compare teams or individuals. Ever. Third, I am much more focused on qualitative than quantitative measures. Where do bad managers come from? http://secretsofconsulting.blogspot.com/2012/02/where-do-bad-managers-come-from.html "Real" executives take the long view. They are the kind of people who, at age 60, can be found planting trees. IT managers who think that fast moving technology requires short-term quick fixes are stuck in a middle management mentality. They will never become real executives. How to Measure Productivity http://managetotest.com/2012/02/27/on-productivity/ That’s why it’s also a bad idea to rely on one metric. Use a combination of tools to focus on areas of concern and improve them. If velocity is going up, but cycle time is staying the same, there may be some problems with story estimation – maybe one voice has started to dominate the process. If output is up, but velocity is down, there may be a problem with prioritization (we’re only working on easy stories). Take the Crappy Path http://www.allthingsquality.com/2012/02/take-crappy-path.html
  • 35. Developers take the Happy Path. Their Unit Tests often consist of using their code in the obvious way in order to produce the expected results. They instinctively want to see their new creations succeed so they inevitably focus on success. See how nicely behaved my code is! Happy, happy. Product Managers take the Happy Path. Put nice values in, take a nice marketing-oriented screenshot of the results. Show it to the Sales Team. Happiness all around. End Users typically take the Happy Path during User Acceptance Testing. A little bit of typical clean input in, typical expected output out. Now let me stop wasting time with this new system and get back to my real work, so I can leave on time and be Happy. Cost-Benefit Analysis of Exploratory Testing in Comparison with Scripted Testing http://www.bth.se/fou/cuppsats.nsf/all/afb1ff655b978c6ac12579660072d996/$file/BTH2011Pang.pdf 7.2 Scenarios in Which SBTM Can Be More Cost-Beneficial Most of the survey respondents and interviewees were of opinion that SBTM is adaptable in any situation of the project. And it could always be cost-beneficial if it is employed in a thoughtful way that is complementary to the testing activities of the development team. But SBTM could be more cost-beneficial when compared with TCBT in the following scenarios. Requirements are vague or insufficient, and not testable They do not have every bit of information about the system Cost constrained projects, for example, when testing is time critical Short iterations Rapid feedback required to the developers High pace There is rapid and constantly change (in requirements, design, etc.) Manual regression testing 7.3 Scenarios in Which TCBT Can Be More Cost-Beneficial Interviewees stated that, in certain scenarios, TCBT is required. When test execution is too expensive or too difficult and what we have is a lot of cheap time to think. If we are working with a test platform that is quite expensive or is available less frequently, it is more cost-effective and makes sense to think about the tests in advance, because what we have is a lot of cheap time to think and expensive time to execute. For example, when we have to use very complex SQL queries to test reports, it is better to prepare it before execution. When there are regulatory requirements Large project or long term project Large project always involves complex settings where the test data needs to be prepared in detail. When they do not have confidence that testers can do good testing without detailed test cases. For example, new testers In traditional mode, there can be a requirement for repeatable test execution When verification of the requirements is required When test cases for automated regression testing are required Management 3.0 in China http://www.noop.nl/2012/02/management-30-in-china.html In Europe I have asked many people for their favorite management practices. But I never expected to find the most inspiring example of Management 3.0 practices in China. 37 Sources for Test Ideas
  • 36. http://thetesteye.com/blog/2012/02/announcing-37-sources-for-test-ideas/ It is often stated, with right, that you should use many, different information sources in order to come up with good test ideas. Learning is Dead http://angryweasel.com/blog/?p=425 Perhaps I’m just being an old fart, but the serendipity of these three events launched a thought that scared the crap out of me. In these days of search engines and Wikipedia, I wonder if anyone knows how to think anymore. Knowledge is much more than learning or regurgitating facts. Many Models – Better Test Ideas http://thetesteye.com/blog/2012/02/many-models-better-test-ideas/ ”Henrik Emilsson has convinced me that skilled software testing is based on invisible mental models that help us see what can be tested. If we can make these visible, we can sharpen our skills, and also teach testing more effectively.” User Interface Test Automation and its Challenges in an Industrial Scenario http://mdh.diva-portal.org/smash/get/diva2:488088/FULLTEXT01 “The growing demand for UI test automation has triggered the development of many tools. Researchers and developers have been continuously working to further improvise the existing approaches. If we look at GUI test evolution we can observe a clear progress from manual testing towards complete automation.” utest Success Story (Interesting story about a stay-at-home dad who makes money as a freelance tester through utest) http://blog.utest.com/thanks-to-utest-guest-post-from-a-gold-tester/2012/02/ ”I am a Gold rated uTester who, after 10 years as a stay-at-home dad, has gone from no income to earning legitimate monthly pay and tremendous professional satisfaction in 3 short months. I felt my story was one worth sharing and hope it both inspires and encourages those in the uTest Community, those considering joining and those involved in creating it.” Word on The Street: 75% of Apps Not Tested http://www.mobileapptesting.com/word-on-the-street-75-of-apps-not-tested/2012/02/ “There’s an obvious reason why that app you just downloaded doesn’t work: it wasn’t tested! In fact, according to a recent article on The Street, that’s the case for 75% of all mobile applications.” Exploring Test Automation http://angryweasel.com/blog/?p=412 ”I’m all for saving time and money, but I have concerns with an automation approach based entirely (or largely) on automating a bunch of manual tests. Good test design considers manual and computer assisted testing as two different attributes – not sequential tasks.” The Secrets of Leadership http://angryweasel.com/blog/?p=422 ”When people get to “Senior” roles at Microsoft, one of the things we expect is an ability to influence without authority, work through others, and basically make those around you better. For our introverted tech geeks, this is often a big wall to climb. Inevitably, one of these Senior non—manager
  • 37. folks tracks me down and asks, “How can I get better at influence?”, or “How can I get others to accept my ideas?”.” Something I did not expect http://www.softwaretestpro.com/Item/5427/Something-I-did-not-expect/ ”Over the past few years, I have been an ardent, arguably minority voice stating my skepticism about the value of GUI-driving test automation, developed by someone other than the developers, after the code is "complete." At the same time, I've admitted that the reality of an increasingly technological society will mean that, while we may not all need to be SDETs, we'll need to be more technical simply to participate. Then something happened that threw me for a loop.” The Skeptic's Dilemma http://angryweasel.com/blog/?p=404 ”But it’s easy to go overboard with skepticism. Time and time again, I hear good testers apply their skepticism broadly and liberally. Some (paraphrased) quotes I’ve heard recently include: “Model-based testing is interesting, but it doesn’t work in a lot of places” “I’m skeptical of static analysis tools – sometimes they have false positives” “Metrics are evil, because someone may use them to measure people” I agree whole heartedly with each of these quotes. However, I worry that folks are throwing the baby out with the bathwater.” "In-The-Wild Testing" (ITWT) http://www.softwaretestpro.com/Item/5422/%22In-The-Wild-Testing%22-The-Missing-Link-in-the- QA-Chain/Test-and-QA-Software-Test-Professionals-Conference http://www.softwaretestpro.com/files/In-The-Wild-Testing.pdf "In-The-Wild Testing" (ITWT) is an effort to educate tech leaders about how to help QA teams and organizations launch higher quality software, quicker, faster, and cheaper. The idea of in-the-wild testing is about providing organizations with the real-world testing data necessary to make informed decisions about releasing products to market. What percentage of testing should be done in a lab, and how much of our testing should be pushed into the wild, to catch issues before the users do?” 8 Things you ought to know if you do not know anything about hiring a software tester http://www.shino.de/2012/01/24/8-things-you-ought-to-know-if-you-do-not-know-anything-about- hiring-a-software-tester/ ”I believe there are some unique skills I would look for in a software tester that I would not necessarily look for in a programmer. So, here it is.” Maybe it is like building a house? http://www.softwaretestpro.com/Item/5386/Maybe-it-is-like-building-houses/ ”A few years ago, I overheard part of a conversation between an academic department on how to teach project management at the graduate level. One professor thought that project management for IT projects is "just like building houses" -- you break down the work into very small chunks, add the chunks up, make a plan, and manage to conformance to plan. The second thought it was more ...
  • 38. nuanced than that. The argument split the department so much that the department chair ended up appointing a third person to teach the slot - one of the few people in the department with a couple of decades of relevant experience. Which brings us to today's question: Is Project Management for software the same as it is for building a house?” That’s a nice theory – How to get people to accept a new approach such as agile http://testobsessed.com/blog/2012/01/21/thats-a-nice-theory/ ” In many organizations agile suggestions fly in the face of their accepted “best practices.” Such ideas also tread on political toes. So one response I hear a lot is: “That’s a nice theory, but it won’t work here.” … The simple reframe from “Why not?” to “What would have to change?” opens up possibilities. What could have become an argument becomes instead a brainstorming session. The result is a chain of steps we can take to go from where we are now to where we want to be.” You are not a complexity thinker when … http://www.noop.nl/2012/01/youre-not-a-complexity-thinker-when.html “You’re not a complexity thinker when… you complain your model is misunderstood or misrepresented by many people because the point of your model should be to enable sense-making.” In Search of Code Quality (there is also a link to a fun comic in here http://www.osnews.com/story/19266/WTFs_m) http://angryweasel.com/blog/?p=402 ”Quality and care are two different ways of looking at the same thing. Quality comes from care, so perhaps what I want to measure is the care put into coding. The fact that a developer wanted to compile at a higher warning level or turn on more static analysis checks, or that they cared enough to get the testers input on testability, or a peer’s advice on maintainability may be the best ways to measure code quality of all.” The Value of Testing http://www.rbcs-us.com/blog/2011/12/26/the-value-of-testing/ ”I received the following message from Helen Huang: ”I am tester .I has work this job of 5 years on China. Now I meet some problem in my working. 1. In china , I find many company always talk the tester value ,what is the tester value ? . eg: Find the number of bug Or find a bug problem ?” This is a common problem.” Testing the Limits with Anne Marie Charrett – Part 1 http://blog.utest.com/testing-the-limits-with-anne-marie-charrett-part-i/2012/01/ ”uTest: In terms of money, time and effort, what is the biggest waste of resources you see in testing today? Certifications? Test reports? Automated tools? You tell us. AMC: Behind this waste lies the concept that there is a simple solution to a complex problem. The above solutions are wasteful when they aim to easily resolve a complex problem. If we could stop looking for the silver bullet, then perhaps these wasteful solutions wouldn’t look so tempting.” Visualising Quality
  • 39. http://gojko.net/2011/04/27/visualising-quality-initial-ideas/ But let’s deal first with the two nihilistic categories. I don’t believe that visualising quality is impossible – it is just difficult. If it were easy we’d all be doing it already, but it is hardly in the same category as flying through space faster than the speed of light. As for the other category, with comments like “it’s between the user and the system, we need to put something in front of the user” etc I disagree. Quality isn’t only between a user and a system, it is between those two and also the people who are charged with delivering software. The Brutal Difficulty of Transformation http://blogs.hbr.org/anthony/2012/01/kodak_and_the_brutal_difficult.html ”2012 has not gotten off to a great start for Eastman Kodak. Three of the company's directors quit near the end of last year, and word recently emerged that the company was on the brink of filing for Chapter 11 bankruptcy protection. The easy narrative is that Kodak is a classic case of a company blind to the disruptive changes in its marketplace. Like many easy narratives, this one is not quite right.” On Test Design http://mdh.diva-portal.org/smash/get/diva2:442409/FULLTEXT01 “Testing is the dominating method for quality assurance of industrial software. Despite its importance and the vast amount of resources invested, there are surprisingly limited efforts spent on testing research, and the few industrially applicable results that emerge are rarely adopted by industry. At the same time, the software industry is in dire need of better support for testing its software within the limited time available. Our aim is to provide a better understanding of how test cases are created and applied, and what factors really impact the quality of the actual test.” Software Testing in Agile Development : Technological and Organisational Challenges http://mdh.diva-portal.org/smash/get/diva2:415961/FULLTEXT02 ”The contemporary industrial trend towards agile software development brings forth new concerns, challenges as well as opportunities. One of the main issues concerns the quality of the final product, for which testing is the well-known assurance mechanism. However, how to perform testing using existing expertise in an agile environment presents a challenging issue for the software industry.” “This result was interpreted as “Respondents would like to use TDD to a significantly higher extent than they actually do presently”.” ”… identified and listed seven potential limiting factors of industrial adoption of TDD: increased development time, insufficient TDD experience/knowledge, lack of upfront design, domain and tool specific issues, lack of developer skill in writing test cases, insufficient adherence to TDD protocol, and legacy code.” Scripted vs. Exploratory Testing http://thetesteye.com/blog/2012/01/the-scripted-and-exploratory-testing-continuum/ ”I have been using the Scripted – Exploratory Testing Continuum in classes to explain how scripted testing and exploratory testing intertwines; and to explain that most of our testing is somewhere in the middle of the continuum.” Model-based Testing – Some Assumptions and Traps
  • 40. http://testers-headache.blogspot.com/2012/01/model-based-testing-some-assumptions.html ”In December I had a very interesting discussion - about model based testing - I was there as a 'customer' and model based testing was being 'sold'. I made my skepticism known before the meeting, by my assertion that all testing is based on models - and that some people are better at observing, constructing, expressing and examining them than others. I also stated I'd seen the power of modeling techniques that allowed testing earlier but that they are far from the complete picture.” Looking for the Great Thought about Management http://www.noop.nl/2011/12/global-management-warming-starts-in-switzerland.html http://www.noop.nl/2012/01/prelude-to-stoos.html http://www.noop.nl/2012/01/about-our-stoos-communication.html http://www.noop.nl/2012/01/stoos-network-part-2-stakeholders-personas.html http://www.noop.nl/2012/01/stoos-network-part-3-core-idea.html ”This Friday and Saturday some 20+ management thinkers and practitioners will gather in Stoos (Switzerland) and talk about management. In particular, we will discuss why management isn’t changing fast enough.” Seven ways to make testing irrelevant on your team http://test.techwell.com/articles/weekly/seven-ways-make-testing-irrelevant-your-team ”Wouldn't it be great if everyone on your team loved testing and working with testers? When developers and testers have a respectful relationship, it's easier for developers to see the value and take an interest in testing. Instead, many testers pick up a knack of alienating themselves—and testing in general—from their team. The following is a list of ways in which testers may make themselves and testing irrelevant to others on their team, along with tips for avoiding getting stuck in these situations.” Top Ten Reasons Why Large Companies Fail To Keep Their Best Talent http://www.forbes.com/sites/ericjackson/2011/12/14/top-ten-reasons-why-large-companies-fail-to- keep-their-best-talent/ “Whether it’s a high-profile tech company like Yahoo!, or a more established conglomerate like GE or Home Depot, large companies have a hard time keeping their best and brightest in house.” A Christmas Gift From a Tester http://www.shino.de/2011/12/24/a-christmas-gift-from-a-tester-for-a-tester-part-one/ http://www.shino.de/2011/12/25/a-christmas-gift-from-a-tester-for-a-tester-part-two/ http://www.shino.de/2011/12/25/a-christmas-gift-from-a-tester-for-a-tester-part-three/ http://www.shino.de/2011/12/25/a-christmas-gift-from-a-tester-for-a-tester-part-four/ http://www.shino.de/2011/12/25/a-christmas-gift-from-a-tester-for-a-tester-part-five/ ”A while back I received a gift from Matt Heusser, my early mentor in the Miagi-Do school. It asked me to test it. So, I decided to make a series of blog entries over the holidays out of it.” Testing the Limits with Richard Stiennon (Security expert) http://blog.utest.com/testing-the-limits-with-richard-stiennon-part-i/2011/12/ http://blog.utest.com/testing-the-limits-with-richard-stiennon-part-ii/2011/12/ uTest: You’ve said before that mobile will not require its own anti-virus systems. That said, it seems that mobile threats are multiplying by the hour. In your view, what’s the biggest security challenge in terms of mobile? RS: Apps, apps, apps. VPNs, firewalls, and carrier filtering are going to impede network based attacks. Containing and vetting applications is the biggest security challenge for the platform vendors.
  • 41. By 2015 the overall threatspace will be ten times worse than it is today. Think about that. There will be TEN breaches as critical as the RSA attack. There will be TEN Google Aurora’s. There will be TEN Stuxnets. There will be 300 thousand new malware variants a day. Management Models, Values and Principles http://www.noop.nl/2011/12/management-models-values-principles.html For example, Peter F. Drucker said there are five tasks for managers: 1. Set objectives 2. Organize 3. Motivate and communicate 4. Measure 5. Develop people Wilful Ignorance on Parade http://www.satisfice.com/blog/archives/680 What a senior tester does at Microsoft http://angryweasel.com/blog/?p=388 Decision Making – senior testers make confident decisions, and do not rely on consensus for making (most) decisions. Influence – Dwight Eisenhower said, “Leadership is the art of getting someone else to do something you want done because he wants to do it.” Senior testers make their team better through influence rather than mandates – they are the point guard of the test team. Network – senior testers use their network to find answers (or knowledge), and connect their teammates with their network (i.e. tester matchmaking) Credibility – senior testers don’t proclaim they’re a leader, and no title demands respect. Leaders know that respect as a leader comes through establishing credibility with their team and peers. For me, the moment you demand respect as a leader is the moment you lose my respect as a leader. Test Levels in Agile Testing http://www.shino.de/2011/12/11/test-levels-in-agile-testing/ Test levels and Test quadrants. How to classify different types of testing. Worthwhile Documentation http://www.developsense.com/blog/2011/12/worthwhile-documentation/ Would you hire a completely incompetent tester who needed to be told absolutely everything, in painful detail?” “We wouldn’t hire anyone like that.” “Fair enough, and I’d hope not. So, why do you insist that people write instructions for them that way?” What Exploratory Testing is Not http://www.developsense.com/blog/2011/12/what-exploratory-testing-is-not-part-1-touring/ http://www.developsense.com/blog/2011/12/what-exploratory-testing-is-not-part-2-after-everything- else-testing/ http://www.developsense.com/blog/2011/12/what-exploratory-testing-is-not-part-3-tool-free-testing/ http://www.developsense.com/blog/2011/12/what-exploratory-testing-is-not-part-4-quick-tests/
  • 42. Michael Bolton tries to define exploratory testing further/more practically by stating what is not Exploratory testing. Status Reporting http://thetesteye.com/blog/2011/12/status-reporting-questions/ ”Status reporting of testing activities is extremely project-dependent. The needs of when and how and what will differ every time. Maybe that’s why there’s so little good writing about status communication; you have to make it up every time.” Why scripted testing is not for novices http://www.satisfice.com/blog/archives/672 ”Scripted test execution is quite a bit more difficult than exploratory testing, unless you are assuming that the tester following the script has exactly the same knowledge and skill as the test designer (even then it is a qualitatively different sort of cognitive process than designing). An exploratory tester is following (indeed forming as he goes) his own intentions and ideas. But, a scripted tester, to do well, must apprehend the intent of the one who wrote the script. Moreover, the scripted tester must go beyond the stated intent and honor the tacit intent, as well– otherwise it’s just shallow, bad testing.” Ten Must-know tips for Performance Testing http://scott-barber.blogspot.com/2011/12/10-must-know-tips-for-performance-test.html ”More than other automation, bad performance test automation leads to: Undetectably incorrect results Good release decisions, based on bad data Surprising, catastrophic failures in production Incorrect hardware purchases Extended down-time Significant media coverage and brand erosion More than other automation, performance test automation demands: Clear objectives (not pass/fail requirements) Valid application usage models Detailed knowledge of the system and the business External test monitoring Cross-team collaboration Unfortunately, bad performance test automation is: Very easy to create, Difficult to detect, and More difficult to correct.” How do I deal with my crappy organization? http://www.noop.nl/2011/12/how-do-i-deal-with-my-organization.html ”It is one of the questions I get most often… How do I deal with my crappy organization? I like my work but I don’t like what management is doing. How do I deal with it? Well, that’s easy…” Agile Alure http://managetotest.com/2011/12/12/151/ ”I got to visit another company’s location this week – Allure Global. They make dynamic signage for movie theaters – concessions and ticketing, etc. They are an Extreme Programming shop, which
  • 43. means they use the XP practices like TDD, collective code ownership, and others. I took a couple of pictures.” The Death of Quality http://trishkhoo.com/2011/10/the-death-of-quality/ ”“The truth is, bad software makes more money” – Goranka Bjedov, STANZ 2011. If quality is dead, what are we doing here? If we are adding value, what is it and how do we measure it?” Selecting Automation Tools http://testobsessed.com/blog/2011/12/01/selecting-test-automation-tools/ If you want a tool to do functional test automation (as opposed to unit testing), you will probably need both a framework and a driver. The framework is responsible for defining the format of the tests, making the connection between the tests and test automation code, executing the tests, and reporting results. The driver is responsible for manipulating the interface. 10 Things about testing that should die http://scott-barber.blogspot.com/2011/11/10-things-about-testing-that-should-die.html ”Face reality testers, neither the product nor the business revolve around you. Think about it. No business, no product, no developers => no need for testers. In fact, if developers wrote perfect code, there'd be no need for you. You are a service provider and your primary clients are the managers, developers, and/or executives. Your secondary clients are product users and investors.” Don’t Forsake Testing in the Rush to Market http://www.mobileapptesting.com/dont-forsake-testing-in-the-rush-to-market/2011/12/ The question if it is better to get an application out early but bugged, and then fix it later, or making good software that is late to the market. Feature Phones still dominate World Market http://www.mobileapptesting.com/feature-phones-still-dominate-world-market/2011/11/ With all the talk of iPhone vs. Android these days, it’s easy to forget how the majority of the world’s mobile users still make calls and access data: via feature phones. A recently released report (download) from mobile strategy firm VisionMobile takes a look at today’s mobile marketplace finding that, despite the sharp rise in smartphone shipments over 2010 and 2011, global smartphone penetration (by OS) is at just 27% API Testing: How can it help http://www.testingmentor.com/imtesty/2011/12/01/api-testinghow-can-it-help/ Key benefits of API testing include: Reduced testing costs Improved productivity Higher functional (business logic) quality Just Fix It! http://angryweasel.com/blog/?p=386 http://chrismcmahonsblog.blogspot.com/2011/12/just-fix-it.html
  • 44. Chris McMahon’s latest post (Just Fix It) proposes that as far as bug tracking goes, the best course of action is to skip the “tracking” part of the workflow and “Just Fix It.” I’m a huge fan of this approach and think that for the most part, tracking a large number of bugs in a big fat bug system (and often overemphasizing the church of the bug curve) pretty much encourage a test-last / test-quality-in workflow. The Future is Mobile Technology http://test.techwell.com/articles/original/future-mobile-technology ”Technology is moving toward a combination of mobility, social connections, and gaming and entertainment. That means we need to be aware of combined technologies, combined software and tools, and combined activities. This requires that we solve new problems with new technology, or different combinations of technology we are familiar with.” Disposable Programs http://secretsofconsulting.blogspot.com/2011/11/disposable-programs.html ”In many IT installations today, the number one problem is program maintenance. Although the total problem is far from simple, there are a number of relatively simple ideas that can be applied immediately to furnish "prompt relief." One such idea is the disposable program.” Shape of Actions http://www.developsense.com/blog/2011/12/shapes-of-actions/ ”In The Shape of Actions, Collins and Kusch describe key differences between two kinds of intentional human actions that they call mimeomorphic and polimorphic. In both words, “morph” refers to shape, or form. “Mimeo-” refers to copying. That is: software development is a polimorphic activity, and if that’s true, testing needs to respond accordingly.” Test is Dead (Test is Dead was the topic of the Keynote of the Google Test Automation Conference this year: http://www.youtube.com/watch?v=X1jWe5rOu3g) On the alleged death of testing http://scott-barber.blogspot.com/2011/11/on-alleged-death-of-testing.html The problem here, at least in my opinion, is simple. Testers have become so focused on legitimizing their discipline and their role that they've forgotten that their role and their discipline exists to serve the companies that employ them. Conversely, the managers and executives of companies that employ testers are regularly directing testers to do these things they don't really understand rumored to be "best practices"-- and of course, then blame the testers when those "best practices" don't provide sufficient value. There is no "basic training" for testers, and there is no "manager basic training" that covers testing and test management as it relates to value to the business -- and even if you believe the training exists, you can't deny that: a) many folks wouldn't agree that training is of value b) even more folks who "should" have received that training haven't even heard of it New Ways of Thinking in Software Testing (Read the comment from Alan Page at the bottom of this blog post)
  • 45. http://www.softwaretestpro.com/Item/5352/New-Ways-of-Thinking-in-Software-Testing/Agile- Software-Test-and-QA-Technology-Strategy-Process-Editorial To put it a different way: If your company makes it's money off of giving away free software and selling advertisements, then, perhaps, if your customers are forgiving, and if you can respond to defects quickly, you can be successful by hiring automated-test-writers (selenium, etc) and crowdsourcing your testing with a product like uTest. The idea here is to have humans find bugs fast - and cheap - perhaps over the weekend - while the automated tools prevent major regressions. And hey, that might just work for google, or for Yahoo, or Mozilla. Would you want your bank, your credit card company, your automobile transmission, or the medical device company that created your pacemaker to use such a strategy? --- Alan Pages comment: I know I don't always play nice with others, but I sort of feel that the test-o-sphere is overreacting to the test is dead story. *If* you're shipping a web service, *and* the web service is mostly non-critical to day-to-day mission critical application, would you pay a tester to test it? Likely, you'd want deep functional testing and some perf load testing, and who knows what else, but it's certainly ok for developers to write this stuff. But I don't think we should take the message so literally. IMO, many testers have been doing developers work (unit & functional) testing for years - and of course it makes sense for developers to do that work. If you have the right set of diagnostic and monitoring tools in place, by all means, get rid of testers. Of course, as you say, if you're testing a medical device, or desktop software, you probably have to modify your story a bit - but to me, I still think there are aspects of the story (e.g. have developers do more testing; or collect customer usage patterns) that can apply to any software project. The reactions (this, and others) are interesting to watch - it's almost as if testers feel that they have to defend their job - but I see the topic as one we can all take something away from. Hell, I've seen some pretty insane ideas come from testers, and I've found some good in nearly all of them. Looking forward for when we can get to Test is Dead (is dead)? Testing the Limits with Noah Sussman http://blog.utest.com/testing-the-limits-with-noah-sussman-part-i/2011/11/ http://blog.utest.com/testing-the-limits-with-noah-sussman-part-ii/2011/11/ uTest: What types of traits/qualities/skills does Etsy look for when hiring testers? NS: On my team there are two roles for which I hire. These aren’t formal roles and we switch them up sometimes. I hire software engineers who value quality and who are interested in how complex systems fail. These are the people who customize our mocks and fixtures, manage our coding standards, build
  • 46. static analysis tools, develop Jenkins plugins, design the CI system in general other things of that nature. … Here I look for talented QA analysts who are longtime Etsy users and deeply involved in the Etsy community. These are the people who develop functional testing tools, manage Selenium integration with CI and work with others in the organization to formulate test plans and resolve bugs. They also design and improve the process by which we triage bugs found in production. Complexity vs Lean vs Agile vs Me http://www.noop.nl/2011/11/complexity-versus-lean-versus-agile.html System thinking in a bar http://www.noop.nl/2011/11/systems-thinking-in-a-bar.html If we want to decide whether or not something must be treated as a system, we first need to consider what part of the world we want to look at, and what questions we want to answer. API Testing – Functional Testing Below the UI http://www.testingmentor.com/imtesty/2011/11/21/api-testingfunctional-testing-below-the-user- interface/ ”Some people assume that API testing is a ‘white-box’ testing activity in which the tester has access to the product source code. But in reality, API testing is truly black-box testing in the truest sense of the testing approach. API testers make no assumptions about how the functionality is implemented, and are not limited by constraints or distracting behaviors of a graphical user interface. ” A tester’s commitments http://www.satisfice.com/blog/archives/652 ”This is the latest version of the commitments I make when I work with a programmer. 1. I provide a service. You are an important client of that service. I am not satisfied unless you are satisfied. 2. I am not the gatekeeper of quality. I don’t “own” quality. Shipping a good product is a goal shared by all of us.” RPF Google’s Record Playback Framework http://googletesting.blogspot.com/2011/11/rpf-googles-record-playback-framework.html ”At GTAC, folks asked how well the Record/Playback (RPF) works in the Browser Integrated Test Environment (BITE). We were originally skeptical ourselves, but figured somebody should try. Here is some anecdotal data and some background on how we started measuring the quality of RPF. The idea is to just let users use the application in the browser, record their actions, and save them as a javascript to play back as a regression test or repro later. Like most test tools, especially code generating ones, it works most of the time but its not perfect.” Activities and Roles http://angryweasel.com/blog/?p=375 ”I’ve been thinking about testing activities and tester roles lately, or more lately, since it seems to be a recurring topic on this blog. I have to admit that I remain a bit surprised how often testers argue about what is and what isn’t testing.”
  • 47. Iterative Development: Some History http://secretsofconsulting.blogspot.com/2011/11/iterative-development-some-history.html http://www.craiglarman.com/wiki/downloads/misc/history-of-iterative-larman-and-basili-ieee- computer.pdf ”We were doing incremental development as early as 1957, in Los Angeles, under the direction of Bernie Dimsdale [at IBM's Service Bureau Corporation]. He was a colleague of John von Neumann, so perhaps he learned it there, or assumed it as totally natural. I do remember Herb Jacobs (primarily, though we all participated) developing a large simulation for Motorola, where the technique used was, as far as I can tell, indistinguishable from XP.” Test Innovation http://angryweasel.com/blog/?p=362 ”Many years ago, test systems at MS reached a level of maturity and scale that enabled teams to easily run millions of tests. These systems[1] were the culmination of several innovations, but eventually led to other problems. The prime example of a problem created by the test system innovation is the challenge of what to do with all of those failed tests. Think for a moment. If you run a million tests and have a 99% pass rate (I know, but this is just an example), you have ten-thousand failures to investigate. Automatic failure analysis is an innovation that greatly reduces the time testers need to investigate automation failures (my stance is that automation is worthless unless every aspect of automation beyond test authoring is not also automated). We’ve also made great strides in test selection (let’s say you only had time to run five thousand of those million tests – which would you run?).” Smoke Test vs. Sanity Check http://www.developsense.com/blog/2011/11/smoke-testing-vs-sanity-testing/ ”The distinction between the smoke and sanity testing is not generally important. In fact, it’s one of the most trivial aspects of testing that I can think of, offhand. Yet it does point to something that is important.” Build in the Cloud: Distributing Build Outputs (Part 4) http://google-engtools.blogspot.com/#!/2011/10/build-in-cloud-distributing-build.html ”In the previous post, we saw how distributing build load across many machines and reusing the results of previous build actions can produce extremely fast build times. This distribution and reuse itself exposed other performance bottlenecks. Specifically, a full build of a large project may produce several gigabytes of output files, all of which need to be shipped from the cloud back to the developer’s local machine. This taxes both our networks and the developer’s local disk, the limited latencies and bandwidth of which slow down the build.” Build in the Cloud: Distributing Build Steps (Part 3) http://google-engtools.blogspot.com/#!/2011/09/build-in-cloud-distributing-build-steps.html “This post builds on these to describe a system for efficiently distributing build steps across a large number of machines. As you will see, details of the source file system and the build system are important to how we achieve fast and efficient distributed builds.” Build in the Cloud: How the build system works (Part 2) http://google-engtools.blogspot.com/#!/2011/08/build-in-cloud-how-build-system-works.html “In this article we will go into a little more detail in how we tell the build system about the source code and its dependencies. In a later article we will explain how we take advantage of having this
  • 48. exact knowledge to distribute the builds at Google across a large cluster of machines and share build results between developers.” Build in the Cloud: Accessing Source Code (Part 1) http://google-engtools.blogspot.com/#!/2011/06/build-in-cloud-accessing-source-code.html ”Much of our day-to-day activities as software engineers involves source code. When we join a project one of the first things we do is look at the source. We want to build it, run it, experiment with changes, test it, and challenge our assumptions about how it works. For most of us this means we start by “checking out” the source from version control. For small to moderately sized projects almost any reasonable version control system is adequate. But as the number of engineers increases and the code base grows, this can put a strain on the version control system and decrease engineer productivity.” Tools for Continuous Integration at Google Scale (The youtube video that was the foundation for the articles above) http://www.youtube.com/watch?v=b52aXZ2yi08 Five orders of Ignorance and Software Testing http://angryweasel.com/blog/?p=348 ”0OI – lack of ignorance. It’s when you know something. I know, for example, that the first track on the Rolling Stone’s Sticky Fingers album is Brown Sugar. 1OI – lack of knowledge. I don’t know (or remember) what the track is following Brown Sugar, but I could find the answer quickly. 2OI – lack of awareness. You have 2OI when you don’t know what you don’t know. I know that there are Stones tunes that I’ve never heard before, but it would be impossible for me to make a list of Stones songs I’ve never heard before. 3OI – lack of process. You have 3OI when you don’t have a suitable method for discovering 2OI (for discovering what you don’t know you don’t know). 4OI – meta-ignorance. You have 4OI when you don’t know about the five levels of ignorance.” ”When we test to ensure requirements were implemented, or the stories function as expected, we are testing for 0OI.” ”Testers also perform a lot of 2OI testing. We explore or run dynamic tests on a system to help discover what we don’t know we don’t know.” Checking Alignment http://testobsessed.com/blog/2011/10/26/checking-alignment/ “How do we know that our Intentions matched the Actual Need, that the Implementation matched our Intentions, and ultimately that the Implementation matched the Actual Need?” ”Finally, if we want to be sure our Implementation matches the Actual Need we have to watch customer behavior carefully. That means monitoring usage statistics …” Testing the Limits with Michael Bolton (Interview) http://blog.utest.com/testing-the-limits-with-michael-bolton-part-ii-2/2011/10/ http://blog.utest.com/testing-the-limits-with-michael-bolton-part-i-2/2011/10/
  • 49. ”uTest: You recently wrote a blog saying that testers should learn to program. Conversely, should programmers learn to test? Why or why not? MB: Programmers should learn to test. They do learn to test. Most programmers attempt to run the code they’ve written and look at it critically to some degree. Great programmers do that testing in a really diversified and yet focused way. What I’d like to get rid of is the idea “I program, I test, and therefore I know all I need to know about testing. I’m good.” To test really well requires study of many ways that things can go wrong, and many ways in which we can be fooled, and many approaches for defending ourselves against them. The best programmers I’ve ever worked with were themselves excellent testers, but they were also aware of their own limitations. They were really respectful of good testers, and, of course, good testers were respectful of them. It was a virtuous cycle.” Should testers play planning poker? http://www.developsense.com/blog/2011/10/should-testers-play-planning-poker/ “Having observed Planning Poker in action, I’m conflicted. Estimating anything is always a bit of a dodgy business, even at the best of times. That’s especially true for investigation and in particular for discovery.” The 3 Criteria for Agile Leadership Practices http://www.noop.nl/2011/10/the-3-criteria-for-agile-leadership-practices.html ”On Twitter I have claimed that a leadership practice is Agile when: 1. It supports people and their interactions; 2. It helps to deliver value to stakeholders; 3. And it does this by improving the system. ” Google Test Analytics - Now in Open Source (James Whittakers ideas of replacing the test plan) http://googletesting.blogspot.com/2011/10/google-test-analytics-now-in-open.html “A group of us at Google set about creating a methodology that can replace a test plan -- it needed to be comprehensive, quick, actionable, and have sustained value to a project.” Behavior-Driven Development vs. Testing (The comments are more interesting than the article) http://www.satisfice.com/blog/archives/638 ”This is that BDD scenario turned into testing: +Scenario 1: Account is in credit+ Given the account is in credit And the card is valid And the dispenser contains cash When the customer requests cash Then check that the account is debited And check that cash is dispensed And check that the card is returned And check that nothing happens that shouldn’t happen and everything else happens that should happen for all variations of this scenario and all possible states of the ATM and all possible states of the customer’s account and all possible states of the rest of the database and all possible states of the system as a whole, and anything happening in the cloud that should not matter but might matter.”
  • 50. Book Review: Perfect Software and Other Illusions About Testing http://www.mkltesthead.com/2011/10/book-review-perfect-software-and-other.html “Some common questions from the preface of the book: Why do we have to bother testing when it just seems to slow us down? Why can't people just build software right, so it doesn't need testing? Do we have to test everything? Why not just test everything? What is it that makes testing so hard? Why does testing take so long? Is perfect software even possible? Why can't we just accept a few bugs?” The Liberation of Testing http://marlenacompton.com/?p=2514 ”In the keynote, Julian talked about how Dr. Barry Boehm has moved on from saying that bugs are found more cheaply earlier in the software process. Instead, Boehm has been writing that it is now just as cheap to fix bugs in production. This was the perfect setup for Keith Stobie’s presentation, “Testing Services in Production”. Keith drove home the point that he is currently much more comfortable testing in production than testing in any type of “staging” environment. Aside from moving testing into a production environment, there is some testing that Keith doesn’t even call testing anymore. He calls it “monitoring.” This is where the wall between testing and production really crumbles away. He talked about moving asserts from tests into the production code. I’m sure that this is nothing especially new. What’s new is that now there are people relying on this code as an indication of system health. The focus is undoubtedly shifting away from looking for bugs and presuming to know what customer’s want to making sure real data is available about system health and that we know how to interpret it.” Complexity thinking http://www.noop.nl/2011/10/complexity-thinking-presentation.html “Here's my latest presentation (that I'm still working on). It is called Complexity Thinking. I did it for the first time this week at the Lean Kanban Central Europe conference, and I will repeat it (with an improved version) at LESS2011.” Take a BITE out of Bugs and Redundant Labor http://googletesting.blogspot.com/2011/10/take-bite-out-of-bugs-and-redundant.html This is one way to empower the manual tester when it comes to browser testing of Crome. Tool- supported manual testing. This was one of the tools James Whittaker talked about when we said that we cannot rely on quantity when it comes to manual testing, we need expert manual testers – and they need support from tools to be effective. Standardized Risks and Charters for Exploratory Testing http://metadoc.sonyericsson.net/login_link.asp?doc=149/03813-LXE1100048&rev=latest This presentation is about standardizing how we work with exploratory testing to make it easier for junior testers to work with it, and to make the results more easily accessible to management and project leaders. Adaptive Software Testing
  • 51. http://metadoc.sonyericsson.net/login_link.asp?doc=151/03813-LXE1100048&rev=latest This presentation is about how to use automated tests together with scripted tests and exploratory testing in one methodology. Using automated tests to explore the risk space at low cost, and then use this information as input to your manual and exploratory testing. Change the environment, not the people http://www.noop.nl/2011/10/change-the-environment.html ”People's behaviors depend on the environment. If you manipulate the environment, you manipulate behaviors.” Work the network http://www.noop.nl/2011/10/work-the-network.html ”Behaviors spread through a social network like viruses. You must work the network to change a whole organization.” Fixing bugs should be the highest priority http://blog.utest.com/fixing-bugs-is-the-highest-priority-period-end-of-story/2011/10/ ”I wanted to share a short but interesting video essay from Scrum co-founder Jeff McKenna on the subject of software bugs – mainly, how we think of them and how we act on them. Here a few items I found particularly intriguing, see if you agree: There’s virtually no difference between high or low priority bugs Fixing bugs is most often based on business factors (i.e. a small bug impacting a big customer) Bug triage is waste of time in terms of learning how to write less bugs Fixing bugs should be the highest priority Unit tests help, but they are not a cure-all Better to find bugs ASAP, to improve the learning” Testing as a whole team activity http://testobsessed.com/blog/2011/09/09/testing-as-a-whole-team-activity/ ”Testing is an activity. Testers happen to be really good at it. We need testers on Agile teams. But if we want real agility, we need to see that completing testing as part of the sprint is the responsibility of the whole team, not just the testers.” And some interesting exercises that could be valuable to try out for any team http://www.noop.nl/2011/09/moving-motivators-free-exercise.html “I invented a new exercise which allows people to reflect on their motivation, and how this is being affected by organizational change.” http://www.noop.nl/2011/09/meddlers-free-exercise.html “The exercise allows players to visualize and discuss organizational structures.” http://www.noop.nl/2011/09/delegation-poker-free-exercise.html “The exercise allows players to discuss how to delegate decisions to a team.” The Dangers of Delegating Discovery – Very interesting article on the dangers of delegating where there is opportunity for unexpected innovation http://blogs.hbr.org/anthony/2011/09/the_dangers_of_delegating_disc.html
  • 52. ”Delegation is a necessary survival skill for senior executives. But when executives delegate their discovery-related innovation tasks, the odds of them finding the surprising insights that often spur transformative-growth businesses decrease dramatically. ” You Can't Analyze Your Way to Growth – Interesting article on that change often cannot be predicted by using historic data http://blogs.hbr.org/martin/2011/09/you-cant-analyze-your-way-to-g.html ”But in the majority of businesses, if the available data are crunched, it shows a slowly growing industry — one growing with GDP or population. That generally convinces the company in question that there aren't really opportunities for top-line growth, and that in turn becomes a self-fulfilling prophecy.” Four Ways to Lead a Successful Transformation http://blogs.hbr.org/cs/2011/09/four_ways_successful_transformation_do.html ” 1. Make the transformation meaningful 2. Be the change you want to see the mindsets and behavior you want to see 3. Build a strong and committed top team 4. Relentlessly pursue impact Managing an organizational transformation, executives tell us, is like trying to change the wheels on a bike while you're riding it. You have to take your organization apart and put it back together in a new way, but you have to keep the business running at the same time. It's a lot to ask, and the senior leader bears much of the burden. ” At least three reasons for testers to learn to program http://www.developsense.com/blog/2011/09/at-least-three-good-reasons-for-testers-to-learn-to- program/ ”Yet there are at least three very good reasons why it might be a good idea for a tester to learn to program. Tooling Insight Humility” Testing Speeds Development http://thetesteye.com/blog/2011/09/testing-speeds-development/ “But the reason I write this post is something Joakim Holm said at his talk, that should be told more often: if you do a lot of testing, development goes faster” The Little Black Book on Test Design http://www.thetesteye.com/papers/TheLittleBlackBookOnTestDesign.pdf “It contains collections of knowledge, and generalizations of my ten years of testing the same product suite. I think it can be useful for ambitious testers that want to find any problems that might be important.” Open Lecture by James Bach on Software Testing http://www.youtube.com/watch?v=ILkT_HV9DVU
  • 53. Google: The 10 Minute Test Plan http://googletesting.blogspot.com/2011/09/10-minute-test-plan.html ”Every time I look at any of the dozens of test plans my teams have written, I see dead test plans. Plans written, reviewed, referred to a few times and then cast aside as the project moves in directions not documented in the plan. This begs the question: if a plan isn’t worth bothering to update, is it worth creating in the first place? To combat this, I came up with a simple task for my teams: write a test plan in 10 minutes. The idea is simple, if test plans have any value at all then let’s get to that value as quickly as possible.” Microsoft: Design for GUI automation http://angryweasel.com/blog/?p=332 ”First off, let me state my main points for disliking GUI automation: It’s (typically) fragile – tests tend to break / stop working / work unsuccessfully often It rarely lasts through multiple versions of a project (another aspect of fragility) It’s freakin’ hard to automate UI (and keep track of state, verify, etc.) Available tools are weak to moderate (this is arguable, depending on what you want to do with the tools).” Test Design for Automation http://angryweasel.com/blog/?p=325 ”A common variation of this is the automation of manual tests. It pains me to hear that some testers design scripted test cases, and then automate those actions. This tells me two things: the manual test cases suck, and the automation (probably) sucks. A good human brain-engaged test case never makes a good automated test case, and automating a scripted scenario is rarely a good automated test (although occasionally, it’s a start). Some teams even separate the test “writers” from the “automators” – which, to me, is a perfect recipe for crap automation.” More Thoughts on Leadership http://www.testingmentor.com/imtesty/2011/08/27/more-thoughts-on-leadership/ ”Leadership is much more than management. A successful leader manages projects by articulating a clear vision, guiding people towards achieving goals, and motivates people by helping them grow. When folks ask me what my first priority is as a Test Lead I say it is the people on my team. But, what does that mean?” Titles for Testers http://angryweasel.com/blog/?p=322 ”I’ve had a few conversations on twitter over the past few weeks about tester titles. My conclusion is that test titles are meaningless, but let’s look at why I think that, and why it doesn’t matter.” Changing People’s Barriers http://www.noop.nl/2011/08/changing-peoples-barriers.html ”The crucial parts of a social system are the people in it. And since all people are different, there’s no one-approach-fits-all solution for change. If you need an organization to improve, you have to work with people’s individual needs, and the various barriers people put up in their minds…” Tomorrow through the Past http://xndev.blogspot.com/2011/08/tomorrow-through-past.html
  • 54. ”This month, on the ASQ blog Paul Borawski notes that perhaps thirty percent, or less, of the attendees at a typical American Society for Quality (ASQ) have heard of W. Edwards Deming, the champion of quality. He asks, in essence "Are we forgetting our history?" I see similar issues in my own tiny sub-field of software testing, mostly in the area of test automation. I have seen organizations implement test automation, expecting test cost to go down ... only to see it actually increase.” Dealing with locale/language specific static test data http://www.testingmentor.com/imtesty/2011/08/21/dealing-with-localelanguage-specific-static-test- data/ ”Many regular readers know that I am a strong proponent of pseudo-random test data generation in conjunction with automated testing to increase the variability of test data used in each test iteration and generally improve test coverage. But I also understand the value of static test data in providing a solid baseline, and in some cases enabling access to specific test data in different locales or languages.” In the Middle http://angryweasel.com/blog/?p=319 ”Should you automate everything, or nothing? Should you test everything, or nothing? How about leadership – should you dictate every detail of what your team should do, or give them no guidance at all. The answer for all of these questions – as you’d expect, is “somewhere in the middle”.” Managing Multiple Bosses http://blogs.hbr.org/hmu/2011/08/managing-multiple-bosses.html ”In the movie Office Space — a comedy about work life in a typical 1990s software company — the protagonist, Peter Gibbons, has eight different bosses. All of them, seemingly unaware of each other, pass by his desk and tell him what to do. While the film is most certainly a satire, for some, it is not far from the truth. More and more people report to more than one boss and learning to handle multiple managers is an essential skill in today's complex organizations. ” Why You Should Automate Parts of Your Job to Save It http://blogs.hbr.org/schrage/2011/08/why-you-should-automate-parts.html ”What is the most important thing you do on your job? What portion of that could be turned into an app that anyone in your organization could effectively use? What portion of that could be automated and fed directly into the larger system with only minimal review by you? What's the least valuable but essential part of your job? Why aren't you figuring out ways to automate it on your iPad or Android?” Exploratory Testing in an Agile Context http://testobsessed.com/wp-content/uploads/2011/08/ETinAgile-agile2011-final.pdf Actually a very good presentation on exploratory testing, and especially chapter 2 about writing charters is very interesting. Recommended read. Technique: Paired Exploratory Survey http://www.satisfice.com/blog/archives/598
  • 55. ”Paired exploratory survey is a process whereby two testers confront one product at the same time for the purpose of learning a product, preparing for formal testing, and/or characterizing its quality as rapidly as possible, whereby one tester (the “driver”) is responsible for open-ended play and all direct interaction with the product while the other tester (the “navigator” or “leader”) acts as documentarian, mission-minder, and co-test-designer.” Re-thinking thinking and planning http://wirfs-brock.com/blog/2011/08/17/re-thinking-thinking-and-planning/ ”In the tutorial, Hooray We’re Agile Testers! What’s Next?, Janet Gregory apologized a couple of times for saying upfront thinking or planning. I know Janet wanted to let the audience know that she isn’t a fan of massive test plans or documents written way ahead…But her remarks got me wondering. Why in the agile community is it a taboo to recommend or admit to doing any upfront thinking or planning?” Who says exploratory testing is good for medical devices? The FDA! http://www.satisfice.com/blog/archives/602 ”This is what I’ve been arguing for a couple of years, now. If you want to test a medical device very well, then you have to test it in an exploratory way. This prepares the way for what the FDA here calls the “pivotal study”, which in software terms is basically a scripted demonstration of the product." Pretotyping: A Different Type of Testing http://googletesting.blogspot.com/2011/08/pretotyping-different-type-of-testing.html ”Testing typically revolves around making sure that we have built something right. Testing activities can be roughly described as “verifying that something works as intended, or as specified.” This is critical. However, before we take steps and invest time and effort to make sure that something built right, we should make sure that the thing we are testing, whether its a new feature or a whole new product, is the right thing to build in the first place.” Criticism against Test Tool Vendors http://www.satisfice.com/blog/archives/596 ”Test tool vendors that bug me: Any vendor who designs tools by guessing what will impress top managers in large companies who know nothing about testing. In other words: tools to support ceremonial software testing.” Effective, Efficient, and Elegant Software Testing http://www.rbcs-us.com/blog/2011/08/03/effective-efficient-and-elegant-software-testing/ ”In chapter 8 *of the Advanced syllabus], I was reading about Standards and Test process improvements. There you talk about Efficiency and Effectiveness. I have one definition about these terms, but what do I have in mind about these terms in order to be prepared for the advanced exam?” The test process and important testing techniques http://uu.diva-portal.org/smash/record.jsf?pid=diva2:428100 ”As software development gets more complex and high paced, the pressure on testing becomes greater to find business critical defects in the software as fast as possible. But testing that is only introduced in the last stages of development will most likely not give any valuable information. The fact is that testing needs to be an active part of development even at the early stages. Requirements need to be written with the
  • 56. input from the testers and testers need informed of what requirements matter, in order to start writing test cases early on. By making the process more effective testers have the chance to increase the quality of both the testing and software.” Testing dojos http://www.shino.de/2011/07/31/testing-dojos-from-the-back-of-the-room/ ”A Testing Dojo is a meeting where testers come together to work on a testing challenge. The testing challenge can consist of testing a product, or generate test ideas for a particular software, or even exercise bug reporting. Mainly the testing challenges will use Exploratory Testing.” Metrics http://thetesteye.com/blog/2011/08/the-metrics-tumour/ ”Testing provides information, so you should find out what is important to different people, and give them what they need, not necessarily what they want. Metrics will hide importance, and also skew the development efforts. Try to take them out, with surgical precision. Then you are left with one final catch: Qualitative information is hard to aggregate, trust is needed.” How Pradeep teaches software testing http://testertested.blogspot.com/2011/07/how-pradeep-teaches-software-testing.html How we tested Google instant pages http://googletesting.blogspot.com/2011/07/how-we-tested-google-instant-pages.html ”How did this testing get organized? As with many things in testing at Google, it came down to people chatting and realizing their work can be helpful for other engineers. This was bottom up, not top down.” // “Anytime test automation scales, is repeatable, quantified, and developers can validate the results without us is a good thing!” Interview with Michael Cooper of T-Mobile http://blog.utest.com/testing-the-limits-with-michael-cooper-of-t-mobile-part-i/2011/07/ http://blog.utest.com/testing-the-limits-with-michael-cooper-of-t-mobile-part-ii/2011/07/ ”uTest: What’s the best way for them to overcome the unique challenges of mobile app testing? MC: I have found that mobile app testing requires a more technical type of testing, and automation is key. We are also looking into crowdsource testing to augment our current testing handset application testing efforts. uTest: What are the biggest pitfalls they should try to avoid? MC: Don’t forget about performance and security testing when working with a partner, or a vendor of mobile testing. uTest: You’ve managed some pretty large testing teams throughout your career. What advice do you have for QA managers whose testing teams are growing beyond 3-5 person teams? MC: I recommend standardizing your testing deliverables and keeping track of metrics so you can manage by facts. Be independent – by that, I mean do everything in your power to not have to report to development. Speak your mind…without whining and complaining. Learn from your mistakes! If you or your team make a mistake or miss a defect, take responsibility and do whatever it
  • 57. takes to ensure that it doesn’t happen again. And, finally, treat your team and your peers with the utmost respect and dignity.” Software Design Engineer in Test vs. Software Test Engineer (Microsoft) http://www.testingmentor.com/imtesty/2011/07/22/the-sdet-vs-ste-debate-redux-its-only-a-title/ ”Testers today face many challenges, and hiring great testers (regardless of the job title) is about finding people who not only have a passion and drive to help improve our customer’s experience and satisfaction, but can also solve tough technical challenges to advance the craft and help improve the company’s business.” Software G Forces: The Effects of Acceleration (Suggested to me as very interesting by Magnus Ernstsson, former DA at PLD SW) http://www.youtube.com/watch?v=KIkUWG5ACFY ”As deployment cycles shrink, what constitutes effective software engineering changes radically. Practices that bring improvement to a quarterly release cycle can be fatal with an hourly release cycle. This talk outlines the changes required of software engineering and organization at different cycle times: quarterly, monthly, weekly, daily, and hourly.” Designing Mobile Apps Using Old School Tools for New School Technology http://www.modernanalyst.com/Resources/Articles/tabid/115/articleType/ArticleView/articleId/1760/ Designing-Mobile-Apps-Using-Old-School-Tools-for-New-School-Technology.aspx ”If we can figure out what we are building and why early on, and support programming by exploring the design quickly, that can save a lot of painful re-work later on. Some old school tools in my Business Analyst design toolbox have turned out to be surprisingly appropriate for mobile application design. I’ll focus on two of them: context-free questions, and state transition networks.” Seven Guidelines For Designing High-Performance Mobile User Experiences http://www.smashingmagazine.com/2011/07/18/seven-guidelines-for-designing-high-performance- mobile-user-experiences/ ”We’ve gone over seven guidelines that address performance relative to different aspects of mobile app design. Design choices affect performance, and so performance should be considered a key factor in the design process. Unfortunately, it tends to be looked at too late in the process, which ends up impairing the user experience significantly.” Testing Tours http://kaner.com/?p=96 ”Tours help the tester gain insight into the multidimensional nature of this complex, infinite space. They help us imagine, envision, and as we gain experience on the tour, prioritize the different sampling strategies we could use when we do more thorough, more intense testing after finishing the tours. So which tour is best? The ones that give the testers more insight and that achieve a greater stretch of the testers’ imagination. For this, some tours will work better for me, others for you.” Testing the Limits with Jim Sivak (Senior QA Manager at McAfee) http://blog.utest.com/testing-the-limits-with-jim-sivak-part-i/2011/06/ ”uTest: In your opinion, is there a law of diminishing returns when it comes to software testing? In other words, after a certain amount of testing, does it really make a difference?
  • 58. JS: Ah, probably the most asked question—are you done testing? There are diminishing returns although my definition might be a bit different than some others. I stop testing when the development team will no longer fix problems because the business has decided that the release must happen. In other words, if bugs that are found are just all deferred, then it is time to stop. uTest: You’ve hired your fair share of software testers over the last 30+ years. What sorts of skills and attributes do you look for when filling out your roster? JS: The best people I hire are the ones who have three key attributes—they love to learn, they constantly look to do things better (not satisfied with the status quo) and they love puzzles. To be a great QA engineer, you have to continuously learn to make yourself and the team better. The best teams continually try new approaches and processes. And testing is solving puzzles—how do you make sure it works, how do you determine what can break and when it does, doing initial looks at what actually happened. I feel that the other skills—programming, product expertise, etc. can be learned on the job.” The Purpose of a Business is NOT customer value http://www.noop.nl/2011/06/the-purpose-of-a-business-is-not-customer-value.html ”The aim proposed here for any organization is for everybody to gain - stockholders, employees, suppliers, customers, community, the environment - over the long term. W. Edwards Deming – New Economics” Dance with the System http://www.noop.nl/2011/06/dance-with-the-system.html ”PDCA, which stands for Plan, Do, Check, Act, is an iterative four-step management process. Any change management initiative consists of a sequence of interventions in a system (or its environment), where you check the effects the interventions have on the system. When your actions work as intended, you move on to the next step. When they don’t work, you try some other ones. It is a never-ending cycle of setting goals (Plan), implementing ideas (Do), measuring feedback (Check) and analyzing results (Act).” Do software testers need a college education http://blog.utest.com/do-software-testers-need-a-college-education/2011/06/ Best practice for Mobile Application Development http://www.53days.com/best-practices-for-mobile-application-development.html ”In order to develop feature-rich mobile applications, there are some key development points which need to be followed upon at the very early development stage. These points will surely enable an IT outsourcing software development to achieve an optimal consumer experience as well as satisfaction.” Failing Fast, 20% Time and Project Mobility http://googletesting.blogspot.com/2011/06/lessons-in-21st-century-tech-career.html ”Failing Fast. Nothing destroys morale more than a death march. Projects going nowhere should do so with the utmost haste. The ability of a company to implode pet projects quickly correlates directly to a great place to work. Engineers working on these project gain not only valuable engineering experience, they experience first-hand the company's perception of what is important (and, in the case of their project, what is not important). It's a built-in lesson on company priorities and it ensures good engineers don't get monopolized by purposeless projects. You gotta like a company willing to experiment. You have to love a company willing to laugh at itself when the experiments don't pan out.
  • 59. 20% Time. Any company worth working for has any number of projects that are worth working on. It's frustrating for many super-sharding engineers to see cool work going on down the hall or in the next building and not being part of it. A day job that takes all day is tiresome. Enter 20% time, a concept meant to send a strong message to all engineers: you always have a spare day. Use it wisely. Project Mobility. Staying fresh by changing projects is part of mobility. Continuous cycling of fresh ideas from new project members to existing projects is another part. The downside here is obviously projects with a steep learning curve but I scoff in the general direction of this idea. Whose fault is it when a wicked smart engineer can't learn the system fast enough to be useful in some (even a small) context? Only the weakest organization with the poorest documentation can use that excuse. The only good reason for keeping people on a project is because they have no desire to leave.” Levels of Testing http://www.testingmentor.com/imtesty/2011/06/25/levels-of-testing/ ”When I teach fundamental testing concepts I often use the ‘levels of testing’ as a model to help students conceptualize different types of testing activities and potentially different stages of testing processes during the software development lifecycle (SDLC).” Automatic Measurement of Source Code Complexity http://www.uppsatser.se/uppsats/93b881462a/ ”The aim of this master thesis is to explore the area of software metrics and to identify software metrics related to the code complexity. In this thesis, thorough study is made to determine whether or not the automatic measurement of source code complexity is possible.” Change Management 3.0 http://www.noop.nl/2011/06/change-management-30.html ”No matter how big the CEO’s desk is, an organization is not a hierarchy. It is a social network. Sometimes you would like to change something in that social network. Sometimes you want people to be nicer, more mindful, or better disciplined. Sometime you just want them to fill out a form, or give you some money. When you want to change behaviors in something as complex as a social system, you have to understand the four aspects of change management.” Fully Automatic Software Testing possible? http://www.shino.de/2011/06/12/fully-automatic-software-testing-now-possible-really-hmm-soooo/ ”One of the fads that seems to be reappearing is the idea of automating away humans from the software development work at all. This fad came up with the rise of UML, and the most recent fad appears to be model-based testing. One of the interesting things I noticed is the ignorance of other past movements. It seems that universities keep on bringing up new talents from time to time that claim to save the world because universities favor a particular competition-based learning model in which everyone wants to be the next hero.” Beyond Agile Programming http://secretsofconsulting.blogspot.com/2011/06/beyond-agile-programming.html Testing at the Speed and Scale of Google http://googletesting.blogspot.com/2011/06/testing-at-speed-and-scale-of-google.html
  • 60. ”Continuous integration systems play a crucial role in keeping software working while it is being developed. The basic steps most continuous integration systems follow are: 1. Get the latest copy of the code. 2. Run all tests. 3. Report results. 4. Repeat 1-3. This works great while the codebase is small, code flux is reasonable and tests are fast. As a codebase grows over time, the effectiveness of such a system decreases. As more code is added, each clean run takes much longer and more changes gets crammed into a single run. If something breaks, finding and backing out the bad change is a tedious and error prone task for development teams.” Basics Revisited: Test Strategy http://www.stickyminds.com/s.asp?F=S16783_COL_2 ”I see many test strategy documents in my work with clients. Almost without exception, they are big—fifty or more pages—and stuffed with soporific boilerplate copied from one project to the next. Defect management process, test suspension and resumption criteria, entry and exit criteria, meaningless generic risks ... yawn ... again. One frequently used model even parrots textbook test definitions, from unit to systems integration to load.” How do you decide what to test? http://angryweasel.com/blog/?p=308 ”The “Test This” game pops up frequently in testing circles. Sometimes it pops up in the “How would you test a …” variety – e.g. How would you test a stapler. There are better flavors of the Test This game. Sometimes, testers play the Test This game with real software. This form has huge potential, but never really delivers, as the software under test is typically a bug-infested piece of crap that anyone with a pulse could find bugs. Bugs are part of my problem with this form of Test This – the goal of this game is to find bugs – not to test or learn testing. That’s an important distinction for me – the goal of testing isn’t to find bugs. Finding bugs is merely a side-effect of the testing process. I fear that these exercises encourage testers to dive into bug finding before thinking about how to test. To be clear, I’m not talking about big-upfront-test-design – but a holistic approach to testing.” Triple Loop Learning http://www.noop.nl/2011/05/triple-loop-learning.html ”The first thing you should do is to gather requirements. What is it that users want your product to be capable of? Do they want a long battery life? A high quality GUI? Automatic updates? A smooth grip? A sturdy case in the color pink? You go out of your way to get a decent list of features to increase your understanding of what is necessary to build the product. It also reduces the risk of welding the foobar to the plingle when users prefer to attach it to a vringle. (I just made that up.)” Tester Respect at Microsoft http://angryweasel.com/blog/?p=305 ”Here’s a tip – you don’t "get" respect, you earn it. In my experience, testers "earn" respect by being credible, showing good results, and making good decisions. You don’t "get" respect from your "level", or by demanding it. Seriously, stop whining and figure out how to earn respect.” Testing the Limits with James Whittaker
  • 61. http://blog.utest.com/testing-the-limits-with-googles-james-whittaker-part-i/2011/05/ http://blog.utest.com/testing-the-limits-with-googles-james-whittaker-part-ii/2011/05/ ”uTest: The world of testing moves incredibly fast at times. That said, is it possible to “future-proof” the theories and methods you write about in your books? Talk about some of the challenges of trying to stay current on the latest testing trends. James: That’s easy. You stay current by inventing the latest trends not following them. Invention is less about being able to read the future and more about a willingness to experiment and be wrong. You should see our drawing room floor at Google, it’s a graveyard of failed experiments. We purposefully try to do things differently to make sure we’re not missing some great new idea or technique. We’ll try almost anything once and we push hard to fail fast. Staying open minded is the key. If you are willing to experiment and expect everything you try to fail, when something actually works, well, you know its something special.” The Illusion of Control http://www.noop.nl/2011/05/the-illusion-of-control.html ”Your eyes capture only a tiny percentage of what happens around you. Your memories are largely just a figment of your imagination. And your “free will” is little more than an elaborate illusion created by your brain. (The Grand Delusion - NewScientist, 14 May 2011)” The Life of a Test Engineer at Google http://googletesting.blogspot.com/2011/05/how-google-tests-software-part-seven.html ”Obviously, the work of a TE varies greatly depending on the project. Some TE’s spend much of their time programming, much like an SET, but with more of a focus on end-to-end user scenarios. Other TE's take existing code and designs determine failure modes and look for errors that will cause those failures. In such a role a TE might modify code but not create it from scratch. TE's must be more systematic and thorough in their test planning and completeness with a focus on the actual usage and system experience. TE's excel at dealing with ambiguity in requirements and at reasoning and communicating about fuzzy problems.” Test Automation – Beyond rudimentary script-lets http://www.testingmentor.com/imtesty/2011/05/19/test-automation-beyond-rudimentary-script-lets/ “In the crud example we basically entered a static “hard-coded” string into a textbox and used a simplistic oracle to make sure the “test data” appeared in a message box. To me, this is about the same as retesting 1 + 1 = 2 over and over again to verify that a calculator program’s addition functionality is computationally correct. IMHO, a “functional test” that verifies a hard-coded string gives great confidence that that string works, but not other strings.” Test Framing http://www.developsense.com/blog/2011/05/ive-been-framed/ http://www.developsense.com/blog/2011/05/framing-test-framing/ ”Framing is the network of logical statements and connectives that link the three story elements and their details. To frame a test means to create an unbroken line of logic from the mission—the information objective within some context—through the product story and our activities, down to the observed outcome of the test and our interpretation of it.” Binary Disease (lots of comments on this article – discussions regarding pass/fail and how to present test results) http://thetesteye.com/blog/2011/05/binary-disease/ ”We make software for computers, and use computers for planning, execution and reporting.
  • 62. Our theories reflect this much more, way much more than the fact that each piece of software is unique, made for humans, by humans. Take a look at these examples: * Pass/Fail hysteria; ranging from the need of expected results, to the necessity of oracles. * Coverage obsession; percentage-wise covered/not-covered reports without elaborations on what is important. * Metrics tumor; quantitative numbers in a world of qualitative feelings. * Sick test design techniques, all made to fit computers; algorithms and classifications disregarding what is important, common, risky, error-prone.” Software Requirements Quality http://www.rbcs-us.com/blog/2011/05/05/software-testing-for-software-requirements-quality/ ”I’m running an ISTQB Advanced Test Analyst course this week in DC. This course includes a hands- on set of exercises based on a realistic requirements specification.” How Google Tests Software – Part Six http://googletesting.blogspot.com/2011/05/how-google-tests-software-part-six.html ”SETs are Software Engineers in Test. They are software engineers who happen to write testing functionality. First and foremost, SETs are developers and the role is touted as a 100% coding role in our recruiting literature and internal job promotion ladders. When SET candidates are interviewed, the “coding bar” is nearly identical to the SWE role with more emphasis that SETs know how to test the code they create. In other words, both SWEs and SETs answer coding questions. SETs are expected to nail a set of testing questions as well.” How Google Tests Software – Q&A http://googletesting.blogspot.com/2011/05/how-google-tests-software-break-for-q.html ”SETs and SWEs are on the same pay scale and virtually the same job ladder. Both roles are essentially 100% coding roles with the former writing test code and the latter doing feature development. From a coding perspective the skill set is a dead match. From a testing perspective we expect a lot more from SETs. But the overlap on coding makes SETs a great fit for SWE positions and vice versa.” Automation isn’t bad – Bad automation is bad http://www.testingmentor.com/imtesty/2011/05/08/automation-isnt-bad-bad-automation-is-bad/ ”The first problem in almost any discussion about automation is the lack of context. Automated tests are different and when done well can be effective at helping us identify some types of functional issues. An automated unit test suite can be a powerful tool for developers during refactoring. Of course some bugs escape unit testing because unit tests are not intended to be comprehensive. I have long been a proponent of functional testing at the API layer and this is where my team plays today. I see first hand the value of our automated tests in the early identification of potentially critical functional issues during our continuous integration process. While this level of testing is effective in helping prevent build breaks, it is not comprehensive and does not test the behavior of the application through the user interface. And then, comes GUI automation.” More on the Coding Tester http://angryweasel.com/blog/?p=299 ”It’s important to note that the role of a coder-tester is NOT to automate everything (everyone knows that you should automate 100% of the tests that should be automated). Writing automation is one task of a coding tester, but certainly not the primary focus (yes, I know that in some companies,
  • 63. automation “experts” only write automated tests. In my opinion, these people are not really testers, so they don’t count).” To automate or not to automate http://angryweasel.com/blog/?p=302 Good testers test first – or at the very least they think of tests first. I think great testers (or at least the testers I consider great) first think about how they’re going to approach a testing problem, then figure out what’s suitable for automation, and what’s not suitable. I say that like it’s easy – it’s not. My stock automation phrase is "You should automate 100% of the tests that should be automated". nearly every tester knows that, but finding the automation line can be tricky business.” Management – The following 30 days http://www.testingmentor.com/imtesty/2011/05/03/the-second-30-days-getting-a-handle-on-the- project/ ”By day I find my time split between people management (team meetings, 1:1’s, etc.) and project management (triage meetings, status meetings, external partner sync-ups, etc.) At night I read through the API specs and brush up on my C++.” Working on Lync http://angryweasel.com/blog/?p=296 ”The "how" part of our work is (to me) much more interesting (and a big reason why I love working on this team). We strive to have a culture high in trust. We have few (none that I can think of, but I’m sure I’m missing something) top-down mandates. We believe that the people closest to the work are in the best place to make decisions on what the work is that they need to do. We obviously give more guidance to employees new to the team and managers are involved in coaching, but for the most part, we trust the people on the team to do what they think is best. For example, we have no requirements on test case counts, bug find rates, code coverage rates, or anything else like that. If someone decides that working on a side project for a few days will help them get their job done better, they don’t need to clear it with anyone – they just do it. Failing is ok (and expected – if you’re not failing, you’re not trying hard enough). ” Ensuring software quality (the first link is a review of the second link) http://angryweasel.com/blog/?p=295  http://www.itmpi.org/assets/base/images/itmpi/privaterooms/capersjones/Chapter5TestOrg.pdf Management – the first 30 days (Microsoft) http://www.testingmentor.com/imtesty/2011/04/11/management-the-first-30-days/ ”It has been more than 8 years since I directly managed a team. In that time some ‘operational’ things have changed, but most importantly I have changed. Personally, I think my responsibilities as a lead include helping the people on my team grow their career, providing a vision and guidance towards clear goals, protecting the team from political fallout and pressure, managing the features(s) I am responsible for, preventing project and personal ‘fires,’ and also actively participating in the testing effort. There are skills, experiences, and values that I have learned in the past that will help me in my transition, there are some adaptations to make, and there is a whole lot to learn.” Management 3.0 http://www.shino.de/2011/04/12/management-3-0/
  • 64. ”Jurgen continued to develop his management model, which consist of six parts – and that formed most of the remainder of the course: Energize people Empower teams Align constraints Develop competence Grow structure Improve everything ” Structure in explortatory testing http://www.developsense.com/blog/2011/04/a-few-observations-on-structure-in-testing/ “When someone says “Exploratory testing is unstructured,” I immediately hear, “I have a very limited view of what ‘structure’ means.”” It’s a software crisis! http://blog.utest.com/its-a-software-crisis/2011/04/ “At least that’s what they were saying in the late 1960s. By chance, I stumbled across a brief Wikipedia entry today on that precise term and suddenly realized that very little has changed in 50 odd years.” Software testing in the development lifecycle (V- W- model) http://www.rbcs-us.com/blog/2011/04/13/software-testing-in-the-development-lifecycle/ ”I think an important thing to remember about these lifecycle models–whether V-model, W-model, Agile model, iterative model, etc.–is encapsulated in a witticism attributed to W.E. Deming: “All models are wrong; some are useful.”” Questions to ask during debriefs http://www.shino.de/2011/04/10/questions-to-ask-during-debriefs/ ”What questions do you ask during an Exploratory Testing session debriefing?” People decide, models inform http://www.noop.nl/2011/04/people-decide-models-inform.html ”Reducing variety is often not the right approach when trying to handle the complexity of the world.” Summary of Swedish Exploratory Testing Workshop 2 http://thetesteye.com/blog/2011/04/thoughts-from-swet2/ Does pair programming obviate the need for code review? http://www.softwarequalityconnection.com/2011/04/does-pair-programming-obviate-the-need-for- code-review/# ”If you're working together with another person to write software, surely you don't need someone else to look over the code. That's part of the process. Or... is it?” Three years of Scrum http://www.softwarequalityconnection.com/2011/03/three-years-of-scrum-at-socialtext/# ”More than an article about Agile transition, this is about actually doing Scrum, with a focus on the daily standup meeting and how it changed over the years.”
  • 65. Software Engineer in Test career path at Google http://googletesting.blogspot.com/2011/04/set-career-path.html ”SETs are developers who write test code and automation as their primary task. They are in every sense of the word a developer. When we interview SETs, SWEs are on the interview loop and SWE questions are asked. They are not all of the interview, but they are part of it.” My job as a tester at Microsoft http://angryweasel.com/blog/?p=293 “I have somewhat of a non-traditional test role at Microsoft. I’m a tester (that’s what my business card says), but I don’t own any specific components or features. I test, but mostly where I want to discover something for myself, experiment with an idea, or when I’m coaching others. The bulk of my time, in fact, is spent coaching and mentoring other testers on the team, as well as working with the test managers on the team and our test director to make sure we’re doing the "right thing" with both strategic and tactical decisions and goals.” It takes complexity to handle complexity http://www.noop.nl/2011/04/it-takes-complexity-to-handle-complexity.html ”In other words, in order to survive a system must have an internal model that reflects the variety it encounters in the world outside.” Questioning test cases: Part 2 http://www.developsense.com/blog/2011/04/questioning-test-cases-part-2/ ”I often hear this kind of justification for test cases: “We need test cases so that testers can learn quickly how to test the product.” To me, that’s like saying that we need driving cases so that people can learn quickly how to drive in a new city. In a new city, a map or a general set of ideas might be helpful. Driving cases won’t, because to drive through a new city well, you have to know how to drive. In the same way, test cases are unlikely to be anything more than negligbly helpful. In order to learn quickly how to test a new program well, you have to know how to test.” Questioning Test Cases: Part 1 http://www.developsense.com/blog/2011/04/questioning-test-cases-part-1/ More of what testers find: Part 2 http://www.developsense.com/blog/2011/04/more-of-what-testers-find-part-ii/ Inspectional testing http://www.shino.de/2011/03/29/inspectional-testing-video/ Dissecting the V-model http://jockeholm.wordpress.com/2011/04/04/dissecting-the-v-model/ What is test control http://www.rbcs-us.com/blog/2011/04/04/what-is-test-control/ Testing the Untestable http://blog.utest.com/testing-the-untestable/2011/03/ “Robert Scoble recently interviewed Trey Ratcliff (one of my favorite photographers) about his new photo editing app for the iPad. Trey remarked that because there is no camera on the iPad 1, they
  • 66. had to “blindly” add the feature for taking pictures using the built-in camera – that is, without testing. That’s because when they wrote the feature, the iPad 2 (which includes a camera) didn’t yet exist. Kind of scary, but understandable. No amount of testing in the world would validate that feature until the iPad 2 went on sale.” A follow up by Michael Bolton on James Bach’s article on what testers find http://www.developsense.com/blog/2011/03/more-of-what-testers-find/ ”How do we know that something threatens the value of the product? The fact is, we don’t know for sure. Quality is value to some person, and different people will have different perceptions of value. Since we don’t own the product, the project, or the business, we can’t make absolute declarations of whether something is a bug or whether it’s worth fixing. The programmers, the managers, and the project owner will make those determinations, and often they’re running in different directions. Some will see a problem as a bug; some won’t. Some won’t even see a problem. It seems like the only certain thing here is uncertainty. So what can we testers do?” It was a big deal – an article about unintended consequences http://xndev.blogspot.com/2011/03/it-was-big-deal.html ”Look, I've been at companies before with a "save the world" mission, I understand the pace of development, and I know how hard it can be to take a few minutes to look around, think, and say something like "Wait ... this feature has some unintended consequences."” Why information is important (Microsoft) http://angryweasel.com/blog/?p=287 ”In the absence of information, people will make stuff up.” ”I was thinking about some of the thoughts and rumors about testing at Microsoft – particularly some of the musings about why MS only hires testers with coding knowledge for full time positions. There are many reasons for this move, and most of them are good reasons – but the role shift hasn’t been without its problems.” What testers find http://www.satisfice.com/blog/archives/572 ”While testing at eBay, recently, it occurred to me that we need a deeper account of what testers find. It’s not just bugs. Here’s my experimental list: Testers find bugs. Testers also find risks. Testers find issues. Testers find testability problems. Testers find artifacts, Testers find curios.” Testing the Limits with Jon Bach http://blog.utest.com/testing-the-limits-with-ebays-jon-bach-part-i/2011/03/ http://blog.utest.com/testing-the-limits-with-ebays-jon-bach-part-ii/2011/03/ ”uTest: Part of your new role at eBay will be to hire and recruit a top-flight team of testers (in addition to the ones already there). What sort of traits/skills/attributes will you be looking for in particular? JB: The ability to come up with ideas – either old or new – and execute them in a way that helps us improve notions of Search. For years, I used the triangle program in test auditions. Now I use
  • 67. something more simple. I draw a long horizontal rectangle on the whiteboard with a little “Submit” button below that. I say “this is a text input field for Search, just like the one you see on the eBay site. Help me create a test plan for it.” I’m hoping that instead of an interview, it comes across more like an invitation to a real collaboration.” Bugs that automated tests aren’t good at finding http://www.testingmentor.com/imtesty/2011/03/27/bugs-that-automated-tests-arent-good-at-finding/ ”Sure, some automated tests can help find bugs, but more importantly it is one approach I might use for defect prevention (esp. computational logic problems), earlier identification of key integration issues, potential degradation of critical areas (battery, performance, memory), efficient execution of redundant ‘checks’ (if necessary or for confidence) more effective/precise ‘oracles’ as compared to humans cost reduction in long term sustained engineering” If you think the primary goal of an automated testing effort is to find lots of bugs, or the same types of bugs as a person then you probably don’t know very much about test automation.” Advanced software testing: Bug isolation http://www.rbcs-us.com/blog/2011/03/27/advanced-software-testing-bug-isolation/ ”Reader Gianni Pucciani has a good question about a question in the Advanced Software Testing: Volume 2 book (regarding bug isolation)” How Google Tests Software – Part 5 http://googletesting.blogspot.com/2011/03/how-google-tests-software-part-five.html ”Instead of distinguishing between code, integration and system testing, Google uses the language of small, medium and large tests emphasizing scope over form. Small tests cover small amounts of code and so on. Each of the three engineering roles may execute any of these types of tests and they may be performed as automated or manual tests.” Testing with Code http://angryweasel.com/blog/?p=286 ”However – if your team only writes regression-ish automation – or if your view of the tester- developer is someone who only writes automated regression tests, you are completely missing the point of coding skills for testers. I know (and fear) that this is the case for way-too-many people in the profession today, and I find it a bit disheartening.” Innovation at Google http://googletesting.blogspot.com/2011/03/innovation-at-google.html ”Patrick Copeland presents the first three principles of the eXtreme innovation approach based on the Pretotyping Manifesto: Innovators Beat Ideas, Pretotypes Beat Productypes, and Data Beats Opinion.” Calculating Defect Detection Effectiveness http://www.rbcs-us.com/blog/2011/03/19/calculating-defect-detection-effectiveness/
  • 68. ”You are a test manager in charge of system testing on a project to update a cruise-control module for a new model of a car. The goal of the cruise-control software update is to make the car more fuel efficient. Assume that management has granted you the time, people, and resources required for your test effort, based on your estimate. Which of the following is an example of a motivational technique for testers that will work properly and is based on the concept of adequate rewards as discussed in the Advanced syllabus? A. Bonuses for the test team based on improving fuel efficiency by 20% or more B. Bonuses for the test team based on detecting 90% of defects prior to release C. Bonuses for individual testers based on finding the largest number of defects D. Criticism of individual testers at team meetings when someone makes a mistake” Are testers made or born? http://www.rbcs-us.com/blog/2011/03/22/are-software-testers-made-or-born/ ”So, is having a team of really good testers and turning them loose without any guidance a good strategy for testing? No.” Technical Debt and Accidental Complexity http://www.jrothman.com/blog/mpd/2011/02/technical-debt-do-managers-unintentionally-force-bad- code.html http://blog.jayfields.com/2011/02/impact-of-accidental-complexity-on.html ”Introducing technical debt increases accidental complexity, and as a side-effect invalidates previous estimates and increases the likelihood that future estimates will differ in size drastically.” Does Quality come from Testing? http://blog.utest.com/does-quality-come-from-testing/2011/03/ ”Do QA professionals truly assure the quality of a product, or do they assist in delivering high quality products?” Simulating your way out of regression testing http://gojko.net/2011/03/03/simulating-your-way-out-of-regression-testing/ ”Julian spoke about alternative testing – or better put alternatives to testing – arguing that time to market is a key competitive advantage for Internet businesses and that exhaustive testing damages that. He cited examples of Facebook, Flickr and Google as companies that do not perform exhaustive regression testing before releases but deal with problems effectively when they appear.” Delegation Poker http://www.noop.nl/2011/03/delegation-poker-game-description.html ”I came up with the idea for Delegation Poker when I tried to create a game that would teach people the following ideas (learning objectives): Delegation is not a binary thing. There are plenty of “shades of grey” between being a dictator and being an anarchist. Delegation is a step-by-step process. You hand over accountability to other people, in a controlled and gradual way. Delegation is context-dependent. You want to delegate as much as possible; but if you go too far, chaos might unfold.” Triangle Test Revisited http://www.softwaretestpro.com/Item/5098 Implementing Lean and Agile software development in Industry
  • 69. http://www.bth.se/fou/forskinfo.nsf/all/45e1d377134f2ac4c12577200040034f/$file/Petersen_diss.pdf You need testers because … http://www.testingmentor.com/imtesty/2011/03/13/you-need-testers-because/ “… testers have a different mindset as compared to developers, and we have different perspectives. Some testers may write, or partner with developers for component level or API tests, but most testers generally focus on integration and system levels of testing. Using various approaches in our craft we often expose functionality issues as well as behavioral or usability issues that might adversely your customers.” Beyond regression tests http://angryweasel.com/blog/?p=284 “In my opinion, an automation strategy that only performs regression testing is short-sighted and incomplete.” “Regression tests, for all the value they bring, tend to train the system to pass.” Battling the Bias http://taooftest.wordpress.com/2011/02/11/battling-the-bias/ ”There seems to be a particular word that crops up again and again in blogs, papers and presentations about software testing and beyond: Bias” How Google Tests Software Part 4 http://googletesting.blogspot.com/2011/03/how-google-tests-software-part-four.html ”One of the key ways Google achieves good results with fewer testers than many companies is that we rarely attempt to ship a large set of features at once. In fact, the exact opposite is often the goal: build the core of a product and release it the moment it is useful to as large a crowd as feasible, then get their feedback and iterate.” Tips on Android Usability Testing http://www.mobileapptesting.com/tips-on-android-usability-testing/2011/03/#more-2305 “… it is important for app developers and testers to focus on usability from the very beginning. An app that is is easy-to-use and intuitive, and similar to industry-accepted interfaces will tend to do well. Usability for mobile apps is not the same for desktop apps: For mobile apps especially the input method, screen size and connection speed differs.” Articles about ISTQB test certificates Certification is evil? http://dorothygraham.blogspot.com/2011/02/part-3-certification-schemes-do-not.html http://dorothygraham.blogspot.com/2011/02/part-2-bit-of-history-about-istqb.html http://dorothygraham.blogspot.com/2011/02/part-1-certification-is-evil.html We don’t need no education – follow up on the above articles about certification http://www.shino.de/2011/03/06/we-dont-need-no-education/ The Optimal Fallacy
  • 70. http://www.noop.nl/2011/03/the-optimal-fallacy.html “There exists a Lean principle called “Optimize the Whole”, which is too often interpreted as an instruction to only optimize “the whole of the system” and to refrain from optimizing subsystems in an organization.” Test Automation: Checking for Bit-ness http://www.testingmentor.com/imtesty/2011/03/03/test-automation-checking-for-bit-ness/ “So, rather than having separate automated tests for 32 and 64 bit operating systems a better test design approach might be to simply detect the bit-ness and auto-magically set variables or redirect the control flow path through your test for the appropriate operating system environment.” Goals, Values, and stuff that makes me mad http://angryweasel.com/blog/?p=282 ”The problem many testers face is that they’re too busy with their "day job" to look up and think about the future. This is as true for looking at the big picture of testing as it is for thinking about their career. I call this "The Tester Treadmill". What I do often in my mentoring sessions is try to help people look up and see a little ways into their future.” Selecting Software Testing Tools http://www.rbcs-us.com/blog/2011/03/06/selecting-software-testing-tools/ ”As software engineers, we want to build useful things, and tools can make us more effective and efficient in doing so. Before we start to use a tool, we should understand the business objectives the tool will promote. Understanding the business case will allow us to properly select a tool. With the tool selected we can then go through one or more pilot projects with the tool, followed by a wider deployment of the tool. As we deploy—and after we deploy—we should plan to measure the return on investment, based on the business case. By following this simple process, you can not only achieve success with tools—you can prove it, using solid ROI numbers.” Problems and Future solutions – Keynote from an Indian test conference http://www.slideshare.net/riarui/bug-debug-keynotepresentproblemsfuturesolutions204 Acceptance Test Driven Development http://testobsessed.com/2011/02/25/the-atdd-arch/ Acceptance Test Driven Development brings teams together http://blog.stephan-schwab.com/2011/02/26/acceptance-test-driven-development-brings-teams- together/ Testing the Limits with Elizabeth Hendrickson http://blog.utest.com/testing-the-limits-with-elisabeth-hendrickson-part-i/2011/02/ http://blog.utest.com/testing-the-limits-with-elisabeth-hendrickson-part-ii/2011/02/ Management 3.0 http://www.noop.nl/2011/02/10-questions-about-management-30-video.html State Transition Testing: Thinking in models
  • 71. http://www.testingmentor.com/imtesty/2011/02/21/state-transition-testing-thinking-in-models/ ”Of course, some people found 1 or 2 issues using an exploratory approach without first creating a state diagram. But, after viewing the state diagram those folks were also able to realize paths they had not considered or traversed and were able to find more issues. The single biggest epiphany that most people have after participating in this exercise is how much more they understand the feature they are modeling compared to what they thought they knew.” The Dual Nature of Context Driven Testing (read the comments for criticism of the article and James’ response) http://www.satisfice.com/blog/archives/565 “Five schools of testing Context Driven: Because testing (and any engineering activity) is a solution to a very difficult problem, it must be tailored to the context of the project, and therefore testing is a human activity that requires a great deal of skill to do well. That’s why we must study it seriously. We must practice our craft. Context-driven testers strive to become the Jedi knights of testing. Agile (the only other school that names itself): sees testing as an adjunct to programming, preferably automated. They accept that exploratory testing is a good thing, but don't feel that there are any special testing skills that deserve or require development. Quality Control: sees testing as an admission of process failure. Someone in the Quality Control school of testing keeps reminding everyone that bugs must be prevented, rather than found. They are testers but don't want to be testers. Analytical (mainly academics): see testing as a fascinating mathematical problem that they are content to solve in well-bounded research exercises, and then hope to scale up. Factory (the biggest school): sees testing as a matter of systematically manufacturing testing artifacts. They focus on documentation, metrics, and detailed instructions intended to control tester behavior. They love automation. T-Map and TPI are great examples of the factory school in action.+” Risk-Based Testing: How fine grained should we be http://www.rbcs-us.com/blog/2011/02/27/risk-based-testing-how-fine-grained-should-we-be/ ”… the question of granularity of the risk analysis is also important. The granularity must be fine- grained enough to allow unambiguous assignment of likelihood and impact scores. However, if you get too fine-grained then the effort goes up to an unacceptable level. A proper balance must be struck.” 4 Ways to Make the Ship No-ship decision http://www.softwarequalityconnection.com/2011/01/4-ways-to-make-the-shipno-ship-decision/# ”In the last decade, I’ve seen a few ways to make the call. I present a four ways I’ve seen for “when to ship” decisions, along with some of the tradeoffs involved with each technique. (And, unlike that textbook, I do it without the big words or fancy symbolic notation, thank you very much.)” Productivity and Projects http://www.kohl.ca/blog/archives/000224.html ”We have a lot to draw on for planning and getting productive, but the concept of staying productive is often lost on us. How a team can sustain value creation over the long haul can be fascinating, and we overlook it at our peril.”
  • 72. This Code if CRAP http://googletesting.blogspot.com/2011/02/this-code-is-crap.html ” As the CRAP acronym suggests, there are several possible patterns that make a piece of code CRAPpy, but we had to start somewhere. Here is the first version of the (in)famous formula to help detect CRAPpy Java methods. Let’s call it CRAP1, to make clear that this covers just one of the many interesting anti-patterns and that there are more to come. CRAP1(m) = comp(m)^2 * (1 – cov(m)/100)^3 + comp(m) Where CRAP1(m) is the CRAP1 score for a method m, comp(m) is the cyclomatic complexity of m, and cov(m) is the basis path code coverage from automated tests for m.” The One That Got Away http://www.stickyminds.com/s.asp?F=S16670_COL_2 “Many testers have been involved in post-ship decisions about bugs that "got away"–bugs that escaped testing and found their way into customer's hands. Often, these post-mortem discussions end up with finger pointing and threats, but with the right focus, these discussions are a wonderful opportunity for learning and growth.” How to test with little to no requirements http://blog.utest.com/how-to-test-with-little-to-no-requirements/2011/02/ ”To wrap things up, software testers need to be agile enough to test under both “planned” and “free” testing environments. And under relatively free environments, testers must draw on creativity, past experience, common sense, and generally accepted practices from similar applications.” – If only our complex reality was this easy … App malware on the rise http://www.mobileapptesting.com/paranoid-android-users-app-malware-on-the-rise/2011/02/ ”If installed, the Trojan gathers the IMEI and IMSI numbers of compromised devices, uploading this information to a remote server, before generating counterfeit queries against particular search results. The malware specifically generated fraudulent clicks on the Baidu ad network, according to anti-virus firm AVG, which reckons the Trojan is the work of a group also producing malware targeting Symbian smartphone.” W. Edwards Deming Some quotes: “It is not necessary to change. Survival is not mandatory.” “Quality is everyone’s responsibility.” "The most important figures that one needs for management are unknown or unknowable, but successful management must nevertheless take account of them." "There is no substitute for knowledge." “Eliminate slogans, exhortations, and targets for the work force asking for zero defects and new levels of productivity. Such exhortations only create adversarial relationships, as the bulk of the causes of low quality and low productivity belong to the system and thus lie beyond the power of the work force.”
  • 73. W. Edwards Deming http://en.wikipedia.org/wiki/W._Edwards_Deming The Wisdom of W. Edwards Deming http://www.youtube.com/watch?v=8jc5fDsgVw0&NR=1&feature=fvwp And there is also a short youtube documentary about him http://www.youtube.com/watch?v=3pmQGTJQ_nc&feature=related http://www.youtube.com/watch?v=PmyieCqE348&feature=related http://www.youtube.com/watch?v=578gaOpT6PU&feature=related The Five Deadly Diseases http://www.youtube.com/watch?v=ehMAwIHGN0Y&feature=related Deming’s 14 Points of Management http://www.youtube.com/watch?v=edE0Cli943k&feature=related How Google Tests Software http://googletesting.blogspot.com/2011/02/how-google-tests-software-part-two.html “The SWE or Software Engineer is the traditional developer role. SWEs write functional code that ships to users.” “The SET or Software Engineer in Test is also a developer role except their focus is on testability.” “The TE or Test Engineer is the exact reverse of the SET. It is a a role that puts testing first and development second.” http://googletesting.blogspot.com/2011/02/how-google-tests-software-part-three.html “… stop treating development and test as separate disciplines. Testing and development go hand in hand. Code a little and test what you built. Then code some more and test some more. Better yet, plan the tests while you code or even before. Test isn’t a separate practice, it’s part and parcel of the development process itself. Quality is not equal to test; it is achieved by putting development and testing into a blender and mixing them until one is indistinguishable from the other.” How we test at Microsoft – The most complete book review of all time http://angryweasel.com/blog/?p=279 Agile Testing Best Practices http://www.rbcs-us.com/blog/2011/01/29/agile-testing-best-practices/ Android surpasses Symbian and Blackberry http://blog.utest.com/symbian-and-blackberry-hangin-tough-as-android-takes-over/2011/02/ Judgement in testing – Part II http://angryweasel.com/blog/?p=278 Why do some testers find the critical problems http://www.developsense.com/blog/2011/02/why-do-some-testers-find-the-critical-problems/
  • 74. Judgement in Testing http://angryweasel.com/blog/?p=277 Mobile Challenges for Project Management, Part 1 http://www.stickyminds.com/s.asp?F=S16589_COL_2 Mobile Challenges for Project Management, Part 2 http://www.stickyminds.com/s.asp?F=S16598_COL_2 How Google Tests Software http://googletesting.blogspot.com/2011/01/how-google-tests-software.html More on Test Design http://angryweasel.com/blog/?p=276 Globlization Testing by BJ Rollison http://www.testingmentor.com/imtesty/2011/01/23/a-source-of-real-world-test-data-for-globalization- testing/ A blog post from Michael Bolton discussing that it is a testers job to not only report bugs, but also provide information regarding risk http://www.developsense.com/blog/2011/01/youve-got-issues/ Fuzzing Through the Side Door by Jonthan Kohl http://www.stickyminds.com/s.asp?F=S16540_COL_2 Authority & Delegation http://www.noop.nl/2011/01/authority-delegation-presentation.html Technical Debt http://thetesteye.com/blog/2011/01/developers-let-the-testers-assist-with-the-technical-debt/ Don’t start a project with scarcity http://www.jrothman.com/blog/mpd/2011/01/dont-start-a-project-with-scarcity.html Career Paths for Testers Using Personas http://www.stickyminds.com/s.asp?F=S16548_COL_2 When a problem isn’t really fixed http://www.developsense.com/blog/2011/01/when-a-bug-isnt-really-fixed/ Agile Management - Complexity Thinking http://www.noop.nl/2011/01/complexity-thinking-presentation.html
  • 75. New Year’s Resolutions by James Whittaker (Google) http://googletesting.blogspot.com/2011/01/new-years-resolutions.html#links Globalization Testing by BJ Rollison (Microsoft) http://www.testingmentor.com/imtesty/2011/01/06/sometimes-bugs-find-you/ Testing Clichés: Testing needs a coverage model (Interesting to think about how his criticism against test coverage models maps against the test coverage model we use) http://thetesteye.com/blog/2011/01/testing-cliches-part-v-testing-needs-a-test-coverage-model/ Effectiveness of Exploratory Testing http://www.bth.se/fou/cuppsats.nsf/all/822127791597b625c12577ea004d36b2/$file/MSE-2010- 24%20(ET%20Thesis).pdf “Using exploratory testing approach testers found far more defects than test case based testing. Although, there was no statistical significance difference between the two approaches for false defects.” “The results drawn from experimental and survey data shows that exploratory testing proved effective and efficient in finding more critical bugs in limited time.” Investigating Exploratory Testing in Industrial Practice http://www.bth.se/fou/cuppsats.nsf/all/955c3ef3da21f788c1257719002bfe6e/$file/MSE%20THESIS% 202010-01%20Final%20Version.pdf “One of the biggest misconceptions of exploratory testing from the customer’s perspective is that it does not provide complete coverage of the system. According to the testers in the study it is possible to completely cover the system testing however due to little or no reliance on structured documentation and simultaneous design and exertion of test cases it is easy to forget the test case or commit mistakes.” “According to the customer perspective the biggest benefit of ET is the ability of providing focused testing. ET facilitates the customers to perform selective but intensive testing according to the project requirements.” Google Test Automation Conference 2010: Videos of all talks http://googletesting.blogspot.com/2010/12/gtac-5-videos-slides-abstracts.html#links Strategic Problem Solving with the 5 Whys (Toyota root cause analysis) http://blog.utest.com/strategic-problem-solving-with-the-five-whys/2010/12/ Testing Challenges & Testing Dojos http://www.shino.de/2010/12/31/testing-challenges/ Manage yourself with measures http://www.noop.nl/2011/01/manage-yourself-with-measures.html ”How We Test Software at Microsoft” Review
  • 76. http://mkl-testhead.blogspot.com/2010/11/testhead-book-club-how-we-test-software.html http://mkl-testhead.blogspot.com/2010/11/testhead-book-club-how-we-test-software_24.html http://mkl-testhead.blogspot.com/2010/11/testhead-book-club-how-we-test-software_28.html http://mkl-testhead.blogspot.com/2010/11/testhead-book-club-how-we-test-software_30.html http://mkl-testhead.blogspot.com/2010/12/testhead-book-club-how-we-test-software_08.html http://mkl-testhead.blogspot.com/2010/12/testhead-book-club-how-we-test-software_13.html http://mkl-testhead.blogspot.com/2010/12/testhead-book-club-how-we-test-software_15.html http://mkl-testhead.blogspot.com/2010/12/testhead-book-club-how-we-test-software_22.html More bang for your testing buck http://blog.utest.com/more-bang-for-your-testing-buck-follow-up-qa-with-james-whittaker/2010/12/ Test Design Technique Name Competition http://thetesteye.com/blog/2010/12/test-design-technique-name-competition/ How to deal with unknown unknowns – Black Swans http://www.noop.nl/2010/12/how-to-deal-with-unknown-unknowns.html No Silos – Combining roles (read the comments as well) http://trishkhoo.com/?p=234 Basic International Sufficiency Testing in Action http://www.testingmentor.com/imtesty/2010/12/16/basic-international-sufficiency-testing-in-action/ The Agile Acid Test http://testobsessed.com/2010/12/14/the-agile-acid-test/ Book Review: Exploring Requirements http://thetesteye.com/blog/2010/12/book-review-exploring-requirements/ Testing the Limits with Matt Evans http://blog.utest.com/testing-the-limits-with-matt-evans-part-i/2010/12/ The biggest difference between a web product and mobile device is the amount of testing and certification that must precede the launch of a mobile product. A smartphone such as the Pre is an incredibly complex and highly integrated piece of technology–much more so than a typical web application. First off, a smartphone contains a fully-functional OS, usually based on some variant of Linux running on very constrained hardware. It must perform all of its concurrent services utilizing limited memory and limited CPU horsepower. The smartphone must also respond correctly to the multitude of many current events, from those generated from the environment–like switching from wifi to a WAN internet connection–to handling data input from the user, as well as handling events from the onboard applications. Launching a mobile product requires exhaustive certification of individual hardware components such as the CPU, modems, codecs, and displays. Even then, the finished product is really launched by the carrier and must go through their exhaustive certification tests as well. Testing an onboard mobile application is also a much harder testing task. There are so many conditions and constraints that are involved in testing a mobile application. A typical mobile application is nearly functionally equivalent to any counterpart desktop client-side
  • 77. web application. Take, for example, a mobile email application. It must behave and interact with the server-side application in nearly the same way as a desktop web client. The established protocols were designed for a stable communications environment, but this is just not the case in a mobile environment. The internet connection may be lost and reconnected very rapidly. The connection may even be lost for long periods of time. The application may, at any moment, be swapped out of memory. The system may be shut down abruptly. Lots of system conditions happen in a smartphone that would rarely or never happen in the context of a desktop web client application. However, a mobile application must perform its main functional operations of retrieving and sending messages flawlessly with no loss of data and full operational integrity. Testing mobile applications under these environmental scenarios is a huge challenge. In short, testing a web application is no easy task, but mobile applications and products represent a much tougher and larger testing challenge. Classification of different tests (integration tests, unit tests, E2E tests, etc) at Google http://googletesting.blogspot.com/2010/12/test-sizes.html#links Exploratory Test Management http://www.shino.de/2010/12/12/eurostar-exploratory-test-management-roundtable/ Generating random dates http://www.testingmentor.com/imtesty/2010/12/07/generating-random-dates/ (Un)common testing insights – A list that according to the author presents “truths” that he thinks are sometimes disregarded http://thetesteye.com/blog/2010/12/uncommon-testing-insights/ Agile transition and employee retention http://testobsessed.com/2010/11/23/agile-transitions-and-employee-retention/ Story of a doomed software business http://www.noop.nl/2010/12/story-of-a-doomed-software-business.html Why is software quality so unmeasured (read the comments as well) http://www.rbcs-us.com/blog/2010/11/28/why-is-software-quality-so-unmeasured/ Coding Dojo – Simple instructions on how a Coding Dojo works Testing Dojo – Applying the coding dojo technique on verification http://www.youtube.com/watch?v=gav9fLVkZQc http://www.shino.de/2010/04/16/testing-dojos/ Combinatorial testing with BJ Rollison http://www.testingmentor.com/imtesty/2010/11/19/combinatorial-testing-complex-interactions/ http://www.testingmentor.com/imtesty/2010/11/25/combinatorial-testing-testing-with-negative-values/ Leadership by Allan Page from Microsoft http://angryweasel.com/blog/?p=252
  • 78. The Drake Equaition of Software Testing http://xndev.blogspot.com/2010/11/drake-equation-of-software-testing.html Synthesizing Test Ideas http://thetesteye.com/blog/2010/11/synthesizing-test-ideas/ Testing the limits with Rex Black http://blog.utest.com/testing-the-limits-with-rex-black-part-i/2010/11/ http://blog.utest.com/testing-the-limits-with-rex-black-part-ii/2010/11/ The Future of Test Management http://www.rbcs-us.com/blog/2010/10/21/the-future-of-test-management/ How outsourcing affects testing http://www.rbcs-us.com/images/documents/howoutsourcingaffectstesting.pdf Return of Investment on Software Testing http://www.rbcs-us.com/blog/2010/09/09/free-tool-for-calculating-software-testing-roi/ Testers: Is it time to reinvent the wheel? – So why are testers expendable? Because we’re not perceived as driving value to the company. And that’s the problem. Next question: How do we go about fixing this? http://blog.utest.com/testers-is-it-time-to-reinvent-the-wheel/2010/11/ Peering into the White Box: A testers approach to Code Reviews http://www.vimeo.com/16753993 Testers doing Code Review http://marlenacompton.com/?p=1966 Areas of Improvement in SW Testing http://angryweasel.com/blog/?p=251 Manage People, Not Roles http://blog.abakas.com/2010/11/manage-people-not-roles.html Engineering Management (Old article from 2009 from a manager at Facebook Engineering) http://algeri-wong.com/yishan/engineering-management.html Hiring Technical People: How long does it take you to hire a candidate http://jrothman.com/blog/htp/2010/10/how-long-does-it-take-you-to-hire-a-candidate.html
  • 79. Combinatorial testing: Testing highly probable combinations http://www.testingmentor.com/imtesty/2010/11/12/combinatorial-testing-testing-highly-probable- combinations/ So, up to now we have discussed: A key to understanding how to use combinatorial or pairwise testing depends largely on our ability to effectively model the input parameters that affect a common output condition We can modify the model to deal with mutually exclusive input value combinations using a simple built in syntax for conditional or invariant constraints We can increase the likelihood of testing different values using random data generation techniques We can increase the test coverage of different combinations by using PICT to randomize the output of the baseline set of tests We can increase the potential to use certain values with greater frequency by weighting important values We can increase the thoroughness of test coverage between specific input combinations with sub-models We can force the tool to include specific input value combinations in a complete n-way output of other combinations using a seed file that specifies those combinations Discussions around the content of a test proposal http://thetesteye.com/blog/2010/10/discussion-around-the-content-of-a-test-proposal/ Software Quality Characteristics 1.0 http://thetesteye.com/blog/2010/11/software-quality-characteristics-1-0/ Turning the Tide of Bad Testing http://thetesteye.com/blog/2010/11/turning-the-tide-of-bad-testing/ Combinatorial testing: Invalid Combinations & Output Condition http://www.testingmentor.com/imtesty/2010/11/03/combinatorial-testing-if-we-cant-predict-an-output- then-we-are-just-guessing/ Forward thinking in Software testing – Read the comments as well, as there is some discussion around the post http://angryweasel.com/blog/?p=239 Combinatorial Testing: Selecting the right values – Part 2 http://www.testingmentor.com/imtesty/2010/10/28/combinatorial-testing-selecting-the-right-values- part-2/ Who owns quality? http://angryweasel.com/blog/?p=232
  • 80. How to win a utest bug battle http://blog.utest.com/how-to-win-the-utest-bug-battle/2010/11/ Combinatorial Testing: Selecting the right values – Part 1 http://www.testingmentor.com/imtesty/2010/10/22/combinatorial-testing-selecting-the-right-values- part-1/ Complexity versus Lean – An interesting 90 page presentation to look through http://www.noop.nl/2010/10/complexity-versus-lean.html A Mnemonic for Mobile App Testing – Article by Jonathan Kohl http://www.kohl.ca/articles/ISLICEDUPFUN.pdf Putting testing to the test – Using crowsourcing for application testing http://www.mobile-ent.biz/features/322/Putting-testing-to-the-test Tester Personas at Microsoft http://msdn.microsoft.com/en-ca/testing/bb414765.aspx Quality in the Test Automation Review Process and Design Review Template http://msdn.microsoft.com/en-us/library/ff521647(v=MSDN.10).aspx Test Automation Code Review Guidelines http://msdn.microsoft.com/en-us/library/ff519670(v=MSDN.10).aspx Rapid Test Preparation http://thetesteye.com/blog/2010/10/rapid-test-preparation/ Testing the Limits with Jeff Papows http://blog.utest.com/testing-the-limits-with-jeff-papows-part-i/2010/10/ http://blog.utest.com/testing-the-limits-with-jeff-papows-part-ii/2010/10/ Estimations Part 3 from Michael Bolton http://www.developsense.com/blog/2010/10/project-estimation-and-black-swans-part-3/ James Bach’s experience report from SWET 1 http://www.satisfice.com/blog/archives/527 Interesting experience report from an test automation conference http://adam.goucher.ca/?p=1621 Thoughts on Test Strategy http://angryweasel.com/blog/?p=228
  • 81. Software Test Estimations VII http://xndev.blogspot.com/2010/10/software-test-estimation-vii.html Two new articles by Michael Bolton on project estimation http://www.developsense.com/blog/2010/10/project-estimation-and-black-swans/ http://www.developsense.com/blog/2010/10/project-estimation-and-black-swans-2/ A tool for managing agile projects – interesting to look through how it works http://www.hansoft.se/ Combinatorial testing http://www.testingmentor.com/imtesty/2010/10/07/combinatorial-testing/ Quality Assurance vs. Test http://angryweasel.com/blog/?p=205 Old Test Plan Template http://strazzere.blogspot.com/2010/09/old-test-plan-template.html 1 million $ worth of bugs http://www.computerweekly.com/Articles/2010/09/30/243093/1m-worth-of-bugs-in-average-app- study-finds.htm Comparing the various crowdsourced testing companies http://blog.utest.com/comparing-the-various-crowdsourced-testing-companies/2010/10/ About Software Engineering – A very interesting lecture regarding software engineering http://xndev.blogspot.com/2010/09/interlude-about-software-engineering.html Test Framing – Michael Bolton and James Bach introduce a new concept http://www.developsense.com/blog/2010/09/test-framing/ Exploratory Testing Best Practices http://thetesteye.com/blog/2010/09/exploratory-testing-best-practices/ Perception of Testing – Future of testing http://www.testingmentor.com/imtesty/2010/09/26/perceptions-of-testing/ Trust and Testing http://angryweasel.com/blog/?p=203 Managers new to agile may not know what to do http://jrothman.com/blog/mpd/2010/09/managers-new-to-agile-may-not-know-what-to-do.html
  • 82. Why exploratory? Isn’t it all just testing? http://www.developsense.com/blog/2010/09/why-exploratory-isnt-it-all-just-testing/ Can exploratory testing be automated http://www.developsense.com/blog/2010/09/can-exploratory-testing-be-automated/ Testing the limits with James Bach – Part 2 http://blog.utest.com/testing-the-limits-with-james-bach-part-ii/2010/09/ Encouraging programmers to be testers http://www.developsense.com/blog/2010/09/encouraging-programmers-to-be-testers/ Exploratory Testing and Review http://www.developsense.com/blog/2010/09/exploratory-testing-and-review/ Test Estimations – part six http://xndev.blogspot.com/2010/09/test-estimation-v_21.html Pradeep blogs about what aspiring testers should think about in India http://testertested.blogspot.com/2010/09/bpo-support-call-center-homemaker-to.html Automated Test Case Design – First step http://www.testingmentor.com/imtesty/2010/09/17/automated-test-case-designthe-first-step/ Competence needed for testing – Part 4 http://googletesting.blogspot.com/2010/09/ingredients-list-for-testing-part-four.html#links What is a Test Strategy? http://angryweasel.com/blog/?p=200 Testing the limits with James Bach – Part 1 http://blog.utest.com/testing-the-limits-with-james-bach-part-i/2010/09/ Exploratory Testing – The Learning Part http://thetesteye.com/blog/2010/09/exploratory-testing-the-learning-part/ Managing focus when doing exploratory testing – An article summarizing a workshop, with links to several presentations regarding exploratory testing – some quite interesting and practical http://www.michaeldkelly.com/archives/508 Agile Backlash – Elizabeth discusses criticism against agile http://testobsessed.com/2010/09/08/agile-backlash/ Definition of Done http://www.developsense.com/blog/2010/09/done-the-relative-rule-and-the-unsettling-rule/ The Complete List of Testing Inspiration
  • 83. http://thetesteye.com/blog/2010/09/the-complete-list-of-testing-inspiration/ On test estimations – 2 more articles on test estimation http://xndev.blogspot.com/2010/09/test-estimation-iv.html http://xndev.blogspot.com/2010/09/test-estimation-v.html The challenge with developers writing automated tests: QA’s perspective http://dev.theladders.com/2010/06/the-challenge-with-developers-writing-automated-tests- qa%E2%80%99s-perspective/ http://www.softwaretestingclub.com/forum/topics/how-do-you-make-sure-that-the - Forum discussion regarding the above article Automation problems – Discussions around that automation often costs more than you think http://angryweasel.com/blog/?p=193 On test estimations http://xndev.blogspot.com/2010/09/on-test-estimation-i.html http://xndev.blogspot.com/2010/09/on-test-estimation-ii.html http://xndev.blogspot.com/2010/09/test-estimation-iii.html The Role of Leadership in Software Development by Mary Poppendieck http://www.youtube.com/watch?v=ypEMdjslEOI Automating Screen Captures http://www.testingmentor.com/imtesty/2010/09/03/automating-screen-captures/ A picture is not worth a thousand words – This article is mentioned in the above article. Talking about how a picture can not replace a good bug report. http://strazzere.blogspot.com/2010/04/picture-is-not-worth-thousand-words.html Putting the pieces together – An interesting discussion regarding testing complex systems http://angryweasel.com/blog/?p=192 Ingredients for testing – Part Three http://googletesting.blogspot.com/2010/09/ingredients-list-for-testing-part-three.html#links The Motive for Metaphor – A discussion regarding the value of using metaphors such as black box and white box testing http://www.developsense.com/blog/2010/09/the-motive-for-metaphor/ 50 Ways to ... Improve Test Automation http://www.stickyminds.com/s.asp?F=S16320_ART_2 Will we survive the future of software http://angryweasel.com/blog/?p=182
  • 84. Are we innovative enough when it comes to software testing? Are we improving how we work as software becomes more complex and harder to test? Are we discussing the same problems today that we were discussing 5 years ago? Introducing Thread-based Test Management – New test management ideas by James Bach http://www.satisfice.com/blog/archives/503 How can I tell users what testers did? http://www.testthisblog.com/2010/08/how-can-i-tell-users-what-testers-did.html Statistician or Journalist – Blog post by Michael Bolton as answer to the above post http://www.developsense.com/blog/2010/08/statistician-or-journalist/ Status of software testing professionals http://thetesteye.com/blog/2010/08/status-of-software-testing-professionals/ Code Coverage: Unreachable Code and Hard to reach code http://www.testingmentor.com/imtesty/2010/08/27/code-coverage-unreachable-code-and-hard-to- reach-code/ The code coverage number is not really useful information to anyone. It is the analysis of the code coverage results that can help us decide whether we need to design additional tests, identify areas of the code that can’t be executed without even more expensive testing such as fault injection and/or code mutation, or refactor the code to improve testability (which often increases the code coverage measure). Ingredient List for testing – Part two http://googletesting.blogspot.com/2010/08/ingredients-list-for-testing-part-two.html#links Testing the limits with Ben Simo http://blog.utest.com/testing-the-limits-with-ben-simo-part-i/2010/08/ http://blog.utest.com/testing-the-limits-with-ben-simo-part-ii/2010/08/ http://blog.utest.com/testing-the-limits-with-ben-simo-part-iii/2010/08/ Why the metric? http://blog.abakas.com/2010/08/why-metric.html All testing is confirmatory http://www.testingperspective.com/?p=428 http://www.developsense.com/blog/2010/08/all-testing-is-not-confirmatory/ - Reply to the above post by Michael Bolton Code Coverage – Finding missing tests http://www.testingmentor.com/imtesty/2010/08/22/code-coverage-finding-missing-tests/ Vilken typ av kompetens en testare bör ha – del 1 http://googletesting.blogspot.com/2010/08/ingredients-list-for-testing-part-one.html#links Why testers need interpersonal skills http://blog.utest.com/7698/2010/08/
  • 85. Relationships matter – Tester & Programmer http://enjoytesting.blogspot.com/2010/08/relationships-matter-tester-and.html Exploratory Testing at Microsoft http://angryweasel.com/blog/?p=177 Code coverage with manual testing http://angryweasel.com/blog/?p=175 How do I create value with my testing? http://www.kohl.ca/blog/archives/000217.html The Testing Planet Magazine http://wiki.softwaretestingclub.com/f/TheTestingPlanet-July2010.pdf No test is also a test http://enjoytesting.blogspot.com/2010/08/no-test-is-also-test-agree.html Careers in test (Microsoft) http://angryweasel.com/blog/?p=163 http://angryweasel.com/blog/?p=164 Code Coverage – Did we run the right tests? (BJ Rollison) http://www.testingmentor.com/imtesty/2010/08/09/code-coverage-did-we-run-the-right-tests/ An approach to code coverage (Microsoft) http://angryweasel.com/blog/?p=172 How to handle regression testing (Bolton) http://www.developsense.com/blog/2010/08/questions-from-listeners-2a-how-to-handle-regression- testing/ Exploring regression testing and product line testing http://www.avhandlingar.se/avhandling/b5a61f2e46/ Search-based approaches to software fault prediction and software testing http://www.avhandlingar.se/avhandling/b61190a783/ Structural System-level Testing of Embedded real time systems http://www.avhandlingar.se/avhandling/d913994e14/ Test Driven Code Review http://googletesting.blogspot.com/2010/08/test-driven-code-review.html#links
  • 86. Kaner on testing http://xndev.blogspot.com/2010/07/kaner-on-testing.html Model-based testing for non functional requirements http://www.uppsatser.se/uppsats/3d3b64a660/ QA activities in Agile http://www.uppsatser.se/uppsats/d1c178735a/ Let’s change the tune – Agile terminology http://gojko.net/2010/08/04/lets-change-the-tune/ Let’s change the title too – Uppföljning på ovanstående artikel av Michael Bolton http://www.developsense.com/blog/2010/08/acceptance-tests-lets-change-the-title-too/ Redefining Done – Definition of Done i Agile https://elearning.industriallogic.com/gh/submit?Action=PageAction&album=blog2009&path=blog200 9/2010/redefiningDone&devLanguage=Java What should done mean http://jrothman.com/blog/mpd/2010/08/what-should-done-mean.html How short can your iterations be http://jrothman.com/blog/mpd/2010/02/how-short-can-your-iterations-be.html Testing lessons from a glass factory http://blog.utest.com/testing-lessons-from-a-glass-factory/2010/07/#more-7311 Model-based testing: An evaluation http://www.uppsatser.se/uppsats/c04c7c4c2e/ Investigating Exploratory Testing in Industrial Practice http://www.uppsatser.se/uppsats/e6e5aed60f/ Testing the limits with Cem Kaner http://blog.utest.com/testing-the-limits-with-cem-kaner-part-i/2010/07/ http://blog.utest.com/testing-the-limits-with-cem-kaner-part-ii/2010/07/ http://blog.utest.com/testing-the-limits-cem-kaner-part-iii/2010/07/ Why test automation costs too much http://testobsessed.com/2010/07/19/why-test-automation-costs-too-much/ Random thoughts on record-and-playback http://testobsessed.com/2010/07/26/random-thoughts-on-record-and-playback/
  • 87. State modeling http://thetesteye.com/blog/2010/06/a-hot-model-that-excites-me/ An analysis of session-based test management http://thetesteye.com/blog/2010/06/an-analysis-of-session-based-test-management/ Behind the scenes of the test group http://thetesteye.com/blog/2010/07/behind-the-scenes-of-the-test-group/ Quality is value to some person at some time http://blog.shino.de/2010/07/22/quality-is-value-to-some-person-at-some-time/#more-805 There, but for the grace of testing, go I http://googletesting.blogspot.com/2010/07/there-but-for-grace-of-testing-go-i.html#links How many errors are left to find? http://www.johndcook.com/blog/2010/07/13/lincoln-index/ Testign stories from developing MacPaint http://blog.utest.com/testing-stories-from-developing-macpaint/2010/07/ To Cert of not to Cert – That is the question http://blog.utest.com/to-cert-or-not-to-cert-that-is-the-question/2010/07/ In-the-lab-testing vs In-the-wild-testing http://blog.utest.com/in-the-lab-testing-vs-in-the-wild-testing-lessons-from-antenna-gate/2010/07/ Testability http://googletesting.blogspot.com/2010/06/testivus-testability-and-dr-jekill-and.html#links Combinatorial tests http://www.testingmentor.com/imtesty/2010/07/01/should-we-use-boundary-values-in-our- combinatorial-tests/ An analysis of session-based test management http://thetesteye.com/blog/2010/06/an-analysis-of-session-based-test-management/ Testing the limits: James Whittaker http://blog.utest.com/testing-the-limits-with-james-whittaker-part-i/2010/06/ http://blog.utest.com/testing-the-limits-with-james-whittaker-part-ii/2010/06/ Agile is not a process; Agile is a mindset http://www.testingmentor.com/imtesty/2010/06/20/agile-is-not-a-process-agile-is-a-mindset/ Is unit testing automated? http://www.developsense.com/blog/2010/06/questions-from-listeners-2-is-unit-testing-automated/
  • 88. Challanging each other helps the craft http://www.satisfice.com/blog/archives/475 Transition to Agile Testing http://www.stickyminds.com/s.asp?F=S16089_COL_2 Some test teams may be stumped on how to transition to agile. If you're in such a team, you probably have manual tests for regression either because you never have had the time to automate them or because you are testing from the GUI and it doesn't make sense to automate them. You probably have great exploratory testers who can find problems inside complex applications, yet they tend not to automate their testing and need a final product before they start testing. You know how to plan the testing for a release, but now everything has to be done inside a two-, three-, or four-week iteration. How do you make it work? How do you keep up with development? Women in Agile http://jonbox.wordpress.com/2010/06/06/context-male-female-or-na/ http://sites.google.com/site/diversityinagile/project-definition#TOC-Program-Mission http://marlenacompton.com/ http://rickscott.dreamwidth.org/3434.html#cutid1 The List is not enough http://thetesteye.com/blog/2010/06/the-list-is-not-enough/ UI automation beneath the presentation layer http://www.testingmentor.com/imtesty/2010/06/09/ui-automation-beneath-the-presentation-layer- using-nets-reflection/ http://www.automatedtestinginstitute.com/home/index.php?option=com_content&view=article&id=12 76&Itemid=122 Experience report: Testing vs. Checking http://testertested.blogspot.com/2010/06/experience-report-of-testing-versus.html http://www.satisfice.com/blog/archives/473 Usability Testing http://thetesteye.com/blog/2010/06/lightweight-usability-testing/ Are testers going away? http://blogs.stpcollaborative.com/matt/2010/05/28/are-testers-going-away/ Technical http://blog.abakas.com/2010/05/technical.html Agile: An experience report http://jonbox.wordpress.com/2010/05/27/what-is-au2h-and-why-i-cared/
  • 89. We can’t find all the important bugs? http://thetesteye.com/blog/2010/06/testing-cliches-part-iv-we-cant-find-all-important-bugs/ Introduction to Test Driven Design http://www.agiledata.org/essays/tdd.html Test Driven Development http://c2.com/cgi/wiki?TestDrivenDevelopment It is better with no model than one model http://thetesteye.com/blog/2010/05/its-better-with-no-model-than-one-model/ Heuristics & Leadership http://www.developsense.com/blog/2010/05/heuristics-and-leadership/ Shipping software is a team effort http://www.testingmentor.com/imtesty/2010/05/24/shipping-software-is-a-team-effort/ Crowdsourcing http://blog.utest.com/to-crowdsourcing-friends-foes-fanatics-just-how-loyal-is-your- community/2010/05/ http://en.wikipedia.org/wiki/Crowdsourcing A new brand of snake oil for software testing – Cem Kaner http://www.satisfice.com/kaner/?p=84 The truth about testing – Jon Bach http://jonbox.wordpress.com/2010/05/19/the-truth-about-testing/ Stuart Reid’s Bizarre Plea by James Bach http://www.satisfice.com/blog/archives/457 Handling inexperienced testers http://www.developsense.com/blog/2010/05/questions-from-listeners-1-handling-inexperienced- testers/ What is a good test case http://thetesteye.com/blog/2010/05/review-of-properties-in-kaners-what-is-a-good-test-case/ Testing the limits with Lanette Creamer http://blog.utest.com/testing-the-limits-with-lanette-creamer-part-i/2010/05/ http://blog.utest.com/testing-the-limits-with-lanette-creamer-part-ii/2010/05/
  • 90. http://blog.utest.com/testing-the-limits-with-lanette-creamer-part-iii/2010/05/ Improve Your Testing and Your Testers with Paired Testing (Test) http://www.informit.com/articles/article.aspx?p=1579370 Automatic Automation (Test) http://blogs.stpcollaborative.com/matt/2010/05/03/automatic-automation/ Measure progress in Scrum by acceptance tests instead of completed tasks (Scrum/Agile) http://www.testingreflections.com/node/view/8489 Without support you will fall (Ledarskap) http://stevenmsmith.com/without-support-you-will-fall/ The nonsense of leadership (Ledarskap) http://www.noop.nl/2010/05/the-nonsense-of-leadership-princes-and-priests.html Reconnaissance – A testing story (Test) http://rapidtester.blogspot.com/2009/10/reconnaissance-testing-story.html Shio or No Ship http://www.developsense.com/blog/2010/05/when-testers-are-asked-for-a-shipno-ship-opinion/ Testers: Get out of the Quality Assurance Business av Michael Bolton http://www.developsense.com/blog/2010/05/testers-get-out-of-the-quality-assurance-business/ How Introverts and Extroverts Perceive Each Other http://www.stickyminds.com/s.asp?F=S16039_COL_2 Testing the limits with Scott Barber http://blog.utest.com/testing-the-limits-with-scott-barber-part-i/2010/04/ http://blog.utest.com/testing-the-limits-with-scott-barber-part-ii/2010/04/ http://blog.utest.com/testing-the-limits-with-scott-barber-part-iii/2010/04/ uTest: Hypothetical: All testers are barred from practicing their craft for an entire year. What does the software industry – nay, the modern world – look like without their services? Please be as dramatic and hyperbolic as possible. SB: Sadly, I think most of the world wouldn’t even notice. I think for the period of a year, non-testers would pick up enough of the slack to keep things moving, and enough individuals would perform enough “heroics” to keep the software world running. Sure, more bugs would make it into production, more software would ship late, and there would be more evening news reports of software failures, but I’m not sure the general public would even notice.
  • 91. In fact, if software producers were smart, they’d pass some of the savings they were realizing by not having to pay testers on to the general public, making them even less likely to mind a few extra bugs. But eventually, some kind of tipping point would be reached and the public would demand less buggy software. That is when the disaster would actually happen. By the time companies re-engaged with testers, re-integrated them into their new (clearly, more streamlined, process), started getting results, realizing just how much re-work needed to be done and how long it would take to get software back up to the “old” standard, the public would be so furious that the “old” standard would no longer suffice. Of course, achieving the new, higher, quality standard would take more time, and make the software cost more, and thus make the public demand yet *more* for their money. Google's Innovation Factory (and how testing adapts) http://googletesting.blogspot.com/2010/04/googles-innovation-factory-and-how.html#links Bättre testdesign ger lägre kostnader http://testzonen.se/?p=1669 Expect the unexpected http://blog.utest.com/eyjafjallajo-what/2010/04/ Software Testers: A costly necessity http://blog.utest.com/software-testers-a-costly-necessity/2010/04/ Exploratory Testing - Is this as good as it gets? http://www.testingmentor.com/imtesty/2010/04/08/is-this-as-good-as-it-gets/ BEhavioral Regression Testing http://people.engr.ncsu.edu/txie/publications/woda08.pdf Five top causes of nasty embedded software bugs. http://www.embedded.com/columns/barrcode/224200699?pgno=1 Complex != Better http://www.testingmentor.com/imtesty/2010/03/31/complex-better/ Working on my first playbook http://annflismark.blogspot.com/2010/03/working-on-my-first-playbook.html Bug Reporting Lessons From Toyota: Are Your Brakes Show Stoppers? http://blog.utest.com/bug-reporting-lessons-from-toyota-are-your-brakes-show- stoppers/2010/03/#more-4706
  • 92. Journey of a passionate tester http://blog.utest.com/journey-of-a-passionate-tester/2010/03/ Boundary Bugs http://www.testingmentor.com/imtesty/2010/03/24/boundary-bugslike-shooting-fish-in-a-barrel/ Management Mistakes http://www.developsense.com/blog/2010/03/management-mistakes-part-1/ Demystifying Exploratory Testing http://www.stickyminds.com/BetterSoftware/magazine.asp?fn=cifea How to reduce the cost of testing http://blogs.stpcollaborative.com/matt/2010/03/11/its-still-interesting/ When Extreme Programming (XP) started in 1999, the rhetoric was that XP teams did not need “QA People” because we were going to automate all the tests. Today, even the founders of XP (Kent Beck, Ward Cunningham, Ron Jeffries) would admit there can be a role for a tester in an XP shop. Now we’re hearing the rhetoric again. How are we supposed to respond? Automated Checks http://blogs.stpcollaborative.com/matt/2010/02/19/the-case-for-automated-checks/ Exhaustive testing http://blogs.stpcollaborative.com/matt/2010/03/15/exhaustive-testing/ Meaningful Measures http://www.testingmentor.com/imtesty/2010/03/21/meaningful-measures/ Interview with Jon Bach http://blog.utest.com/testing-the-limits-with-jon-bach-part-i/2010/03/ http://blog.utest.com/testing-the-limits-with-jon-bach-part-ii/2010/03/ JIRA Defect Tracking System http://www.testinggeek.com/index.php/testing-tools/test-management/177-jira-defect-tracking-system James Bach http://www.satisfice.com/blog/archives/420 : Toyota http://www.satisfice.com/blog/archives/418 : Indian Testers
  • 93. Joel on Software goes Offline http://www.inc.com/magazine/20100301/lets-take-this-offline.html The Customer is always right http://blog.utest.com/the-client-is-always-right-especially-when-theyre-wrong/2010/03/ Do I really need to automate this test? http://www.testingmentor.com/imtesty/2010/03/08/do-i-really-need-to-automate-this-test/ Lessons Learned from Toyota http://blog.utest.com/testing-lessons-learned-from-toyota/2010/02/#more-4114 Interview with James Bach http://blog.utest.com/testing-the-limits-with-james-bach-part-1/2009/12/ http://blog.utest.com/testing-the-limits-with-james-bach-part-2/2009/12/ Interview with Michael Bolton http://blog.utest.com/testing-the-limits-with-michael-bolton-part-i/2010/01/ http://blog.utest.com/testing-the-limits-with-michael-bolton-part-ii/2010/01/ http://blog.utest.com/testing-the-limits-with-michael-bolton-part-iii/2010/01/ Interview with James Whittaker http://blog.utest.com/testing-the-limits-with-james-whittaker-part-one/2009/06/ http://blog.utest.com/testing-the-limits-with-james-whittaker-part-two/2009/07/ Interview with Patrick Copeland http://blog.utest.com/testing-the-limits-with-patrick-copeland-part-i/2010/02/ http://blog.utest.com/testing-the-limits-with-patrick-copeland-part-ii/2010/02/ http://blog.utest.com/testing-the-limits-with-patrick-copeland-part-iii/2010/02/ http://www.developsense.com/blog/2010/02/ellis-island-bug.html http://www.developsense.com/blog/2010/02/return-to-ellis-island.html Scary stories and GUI automation http://www.testingmentor.com/imtesty/2010/02/17/scary-stories-and-gui-automation/#comments
  • 94. Lines of Code and Tests in Balance http://binstock.blogspot.com/2010/02/keeping-loc-and-tests-in-balance.html#links The Role of the Test Manager in an Agile Organization http://www.stickyminds.com/s.asp?F=S15833_COL_2 http://googletesting.blogspot.com/2010/02/testing-in-data-center-manufacturing-no.html#links http://www.developsense.com/2010/02/testing-and-management-parallels.html http://testertested.blogspot.com/2010/02/coaching-testers-on-bug-reports.html http://thetesteye.com/blog/2010/02/more-definitions-of-quality/ http://www.testingmentor.com/imtesty/2010/02/02/api-testing-testing-in-layers/ Burning Issues of the Day av Michael Bolton http://qualtech.newsweaver.ie/startester/ep56r0pdolk-5clln9zu2r An Overview of “A Visual Approach to Risk-Based Integration Testing av Neil Pandit http://qualtech.newsweaver.ie/startester/1l1hwqembqu-5clln9zu2r Logging: Exploratory Tester’s Friend http://www.satisfice.com/blog/archives/401 Exploratory Testing is accountable http://www.developsense.com/2010/01/exploratory-testing-is-accountable.html Code Coverage – More than just a number http://www.testingmentor.com/imtesty/2010/01/21/code-coverage-more-than-just-a-number/ http://www.testingmentor.com/imtesty/2009/11/13/the-code-coverage-metric-is-inversely- proportional-to-the-criticality-of-the-information-it-provides/ http://www.testingmentor.com/imtesty/2009/11/25/reconsidering-code-coverage/ ”there is no correlation between code coverage and quality, and code coverage measures don’t tell us “how well” the code was tested”
  • 95. How to create test cases from bugs http://www.testinggeek.com/index.php/testing-articles/138-testcase-pattern-from-bugs http://www.testinggeek.com/index.php/testing-articles/187-wrong-reasons-to-be-in-testing-field Critical Mass of Software http://www.testingreflections.com/node/view/8429 Limitations of TDD http://binstock.blogspot.com/2009/11/limitations-of-tdd.html http://www.developsense.com/2010/01/disposable-time.html http://testertested.blogspot.com/2010/01/test-effort-estimation-getting-it-wrong.html#more http://www.testingmentor.com/imtesty/2009/12/29/thinking-about-critical-thinking-and-test-design/ http://www.testingmentor.com/imtesty/2009/12/10/evaluating-exploratory-testing/ http://www.developsense.com/2010/01/defect-detection-efficiency-evaluation.html http://www.developsense.com/2009/12/handling-overstructured-mission.html http://www.satisfice.com/blog/archives/398 http://shrinik.blogspot.com/2009/12/why-gui-test-automation-is-popular-and.html My experiences in testing with an exploratory mindset and methodology http://selenadelesie.com/2009/12/18/charting-a-course/ Be an explorer http://selenadelesie.com/2009/12/16/be-an-explorer/ Structures of Exploratory Testing http://www.developsense.com/2009/12/structures-of-exploratory-testing.html
  • 96. Best Bug or Bugs http://www.developsense.com/2009/12/best-bug-or-bugs.html http://blogs.harvardbusiness.org/hbr/cramm/2009/08/are-we-failing-theory-or-is-th.html http://blogs.harvardbusiness.org/hbr/cramm/2009/09/how-are-you-turning-best- pract.html?cm_mmc=npv-_-MANAGEMENT_TIP-_-NOV_2009-_-MTOD1119 http://www.n2growth.com/blog/the-downside-of-best-practices/ http://weo-cuttime.blogspot.com/2009/08/on-best-practice-or-not.html _____________________________________________ Moving to an Agile Testing Environment: What Went Right, What Went Wrong http://stickyminds.com/Media/Video/Detail.aspx?WebPage=165 ”Why is testing taking so long” http://www.developsense.com/2009/11/why-is-testing-taking-so-long-part-1.html Weekend Testing http://weekendtesting.com/ Agile, Testing, and Quality: Looking Back, Moving Forward av Elizabeth Hendriksson http://testobsessed.com/wordpress/wp-content/uploads/2009/10/atqlblf-pnsqc2009.pdf Test Management in an Agile organization av Elizabeth Hendriksson http://testobsessed.com/2009/10/06/specialized-test-management-systems-are-an-agile-impediment/ Fedex Tour http://googletesting.blogspot.com/2009/10/fedex-tour.html#links ”How to get started with TDD” http://googletesting.blogspot.com/2009/11/how-to-get-started-with-tdd.html#links _____________________________________________ James Whittaker on the Future of Testing http://blog.testlabs.com/2009/09/james-whittaker-on-future-of-testing.html
  • 97. James Bach http://www.satisfice.com/blog/archives/370 som länkar till http://www.satisfice.com/blog/wp- content/uploads/2009/10/et-dynamics22.pdf Michael Bolton http://www.developsense.com/2009/10/maturity-models-have-it-backwards.html http://www.developsense.com/2009/10/comment-on-not-so-good-article-on.html Aaron Evans http://fijiaaron.wordpress.com/2009/09/23/verification-and-exploration/ Scott Barber http://www.testingreflections.com/node/view/8282#comment _____________________________________________ Exploratory Testing Dispute http://googletesting.blogspot.com/2009/10/star-west-trip-report.html

×