Thursday, December 30, 2010

Welcome 2011


2010 is almost over. Looking back I see changes. Like the famous quote "nothing is permanent but change" I see change more so in 2010 than in previous years. I adopted a lot of terms, terms I knew but didn't understand, terms I didn't know and now its my everyday word. So here is a small summary:
  • I started blogging here and now write regularly in a couple of different places
  • I made hundreds of tester friends online: people who motivate me on a daily basis, who teach me everyday and who help me be a better person
  • I came to realize I have a lot to learn
  • IPad, Xbox, Android, Twitter, Facebook: tools to help me if I learn to use them
  • I said goodbye to simple flip phones, home phone, GPS, CD player, big huge TV and computer monitors
I welcome 2011 with the following resolutions
  • I want to present at a conference
  • I want to get published
  • I want to travel to Europe
  • I want to go on a cruise
  • I want to complete my green belt certification
With these words I sign off in 2010 to return fully charged in 2011.

Monday, December 27, 2010

Musings - Sherlock Holmes

A friend of mine sent me link to PBS Masterpiece Mysteries where there were links to the new Sherlock Holmes series. This is where Sherlock Holmes meets 21st Century CSI. This show is brilliant and I loved it.


When I was watching the show it reminded me of testing. Holmes strengths are attention to details and observation (investigation). He observes and lets his observation tell him things. Things (requirements or bugs in our world) that others might miss easily because either they were not looking for it or didn't know to look for.

It’s the same with testing - we are observing and looking for things that are out of place (bugs) and things that should be there (requirements). When we find bugs, we should build the story. Build it: where could it have originated? What did we do to get here? What values can cause the system to break? What else can cause the system to break?

We can help the rest of the team (software engineers or business analysts) if we went a few steps further than just logging issues. We could give them as much information as possible. All we have to do is look for it, observe it and document it. We have the information (we saw the bug first, have the test cases, know the requirements, etc) and we have the system where the bug was found, let’s now go and build the story if possible by asking the questions and trying to answer them.

With New Year right around the corner let’s try and be better investigators in 2011. Let’s find the bugs before the customers do and let’s show our team we are Sherlock Holmes the best software detective in town.

Happy New Year Friends.

Friday, December 17, 2010

Friday's Musings - Process Improvement

For the past few weeks I have been busy trying to find a project for my six sigma training scheduled in Feb. I have been looking at various processes within our testing team to see which one could be a good candidate for my project. This is not as easy as I thought it would be.

Yes we see a lot of broken processes and we think something is broken, needs improvement, needs help, needs fixing, etc. But when we starting digging deeper into the issues we see that what we think is the problem is not always the real problem. The problem maybe visible at certain points of the process but in reality the domino effect started somewhere else and the real trick is to find that origin of the problem.

So before we start digging we really have to do the ground work. Its not always easy to say yes I want to reduce the time it takes for testing an application. Well then how much time should it take? Who decides that? Same with we find too many defects or too less defects... well then who defines how many defects is good enough?
I don't have the answer to these questions and I don't have a project defined yet. I will be working the next few weeks to define it and I am sure my readers will hear about it. So for now I leave you all with
  1. Problems are not easy to define, especially if its gut feelings and not real data.
  2. Everyone has an opinion on why things are broken but as six sigma practitioners we have to learn to take the weed out and look at the data for answers.
  3. Questions lead to more questions and there is no easy way out.
Signing off with the hope 2011 will give me more answers than questions for my project.

Sunday, December 5, 2010

My Take: HPQC 11/ALM 11

Recently I had a chance to attend seminars for TFS Test Manager and HP Quality Center 11. Its interesting to see that HPQC is jumping into the bandwagon of having complete software development life cycle in one tool. They were known for test and defect management only. They biggest threat is TFS Test Manager which includes project, requirements, builds, test environment management and test management all in one tool.


With QC 11/ALM 11 HP is trying to catch up to competition. So from just managing testing pieces, they are moving into application life cycle management. This is what HPQC users have been asking them to do for a longtime.

As a tester I am excited about
  1. Record and play back - yes in QC you will finally be able to record what testing real time.
  2. Dashboard that will show scope of a release, requirements , test cases and defects associated with each release. We had to run reports outside of QC to gather this information.
  3. Rich text editor - which will save a lot of time for us testers who add notes outside QC and then attach them here.
  4. Test case re-usability - We can reuse test cases for various configurations and assign it to multiple builds and environments.
  5. Exploratory testing and mirror testing - we can now create test scripts from recording our exploratory test cases. You can testing multiple test environments at the same time using mirror test cases.

Friday, December 3, 2010

Friday's Musings - Reduce Redundancy

Where do we typically see redundancy? Test cases and test plan.

We have to try to reduce this redundancy for the following reasons
  • Waste of time - if we are repeating the same step or steps in more than one test case we are loosing time that could be used for actual testing or other activities.
  • Painful to update and change - if one of those steps that is common in a couple of test cases changes, we have to manually go through all the tests to change them
What should we do?
  • Create call tests or test templates when you know you have a bunch of steps that will be common in quite a few test cases
  • Create a matrix to make sure you are not over testing. You only need tests to cover requirements and if you can combine requirements in a test case, go ahead and do it.
  • Use automation if possible to get some of the redundancy out of the way.
  • For test plan if there are sections that you repeat release after release, think about pulling it out into a handbook or a master test plan where you can maintain this in one place for all release. Have change control on the document to indicate changes when its applicable.

Thursday, December 2, 2010

Help with an Article I am Working on

I am doing a survey for an article I am working on. If you would like to participate please reply to the following question and add your name and occupation at the end. Send the reply to shillu13@gmail.com or post it as a comment here.


Name one technology/trend that you adopted in 2010? (IPad, ITouch, Ebook reader, touch screen phone, cloud services, agile process etc) Do you love or hate it? Would it be hard to give it up? What other tool or technology did it replace? (old flip phone, desk top, books, waterfall etc)

Your help is much appreciated.

ITKE - The IT File Series - Yusuf Salwait

I will be talking with Yusuf Salwait soon. If you have any questions for him please post them here or at the following link.

http://itknowledgeexchange.techtarget.com/software-testing-big-picture/the-it-files-yusuf-salwati/

"He is a blogger @ITKE IT Project Management since sept 2007. In his blog he has written on various topics related to emerging trends, technology changes and trends, project management and current topics related to IT. Yusuf’s background is in the IT and IT management. He has been in the IT field for almost 15 years and started working with customer issues with major computer manufacturers and then moved on to work as programmer, tester and now manager.


His area of expertise is in working with customer service related issues within IT Industry. Also he has worked several different countries and can answer questions related to different in cultural understanding of IT terminologies."

Thursday, November 25, 2010

Friday's Musings - Heads up you guys: sending bugs your way

As a tester, when we find bugs we are trained to log them. There are industry standards out there on how to write a good bug and what details need to be included. What we forget after we log a defect is: Now what?

What I always do is call or email my software engineers. No not to show off or blow my horns but to give them a "heads up". I always call them if its the right time (have a few offshore and wait till its a good time for them) and tell them this is what I found, this is what I was doing and this is what happened (especially if its high or critical issue).

Why do this?
  1. The heads up gives them a chance to react to it before the bug triage/defect review meeting. You are giving them an opportunity to research it and be ready when questions are asked in meetings. This way your team is already a step ahead in trying to answer the questions like "How critical is it?" or "How easy is it to fix it?" etc.
  2. When we give them a heads up immediately they may ask you to look for additional information and you will still have the system available to help them trouble shoot. If its days later and if you have had builds deployed or other test data changes you may not be able to help them replicate defects as easily as you would want to.
  3. You are being a team player. You are not catching them by surprise in review meetings and they will return the favor by sharing information and helping you to when needed.
  4. Gives an opportunity to learn. Several of these calls to my developers have lead to more questions and then into discussion sessions to trouble shoot. They ask for details and information and this has helped me take notes on things that they are looking for in bugs. I use these tips/notes to write better bugs. So the next time I call them I have answers to a few more questions that I had the first time I called them to report a bug.
  5. You are saving time. Instead of waiting till the defect or bug get to the review meeting and then being assigned for analysis and then wait till another meeting to decide if the bug has to be fixed or not you are giving opportunity to react in the first meeting itself with information from software engineers.
I am tagging this under Change Leader. Why? Because you can start small changes and lead the team to do better. I call these small changes as signs to build better teams and as a result better products for our customers: internal and external.

Do you have tips on saving time, building teams or help testing team, please share?

Tuesday, November 23, 2010

2 Things To Make Test Leading Job Easy

As test leads we have several responsibilities between testing, manging project, testing, test planning, etc. You can do these two things daily to make your life more manageable and easier.

Keep a Test Journal
Clean your data everyday 

Read more at 
 http://itknowledgeexchange.techtarget.com/software-testing-big-picture/2-things-to-make-test-leading-job-easy/

Monday, November 22, 2010

My Take: Test Manager (TFS Visual Studio 2010)

I was at the Microsoft Event - The Full Testing Experience - Quality Assurance with Visual Studio 2010. I am pretty excited about this tool and the demo I saw. There are some cool features that I wouldn't mind using to test my application. The one thing that was preventing me from using this tool so long (even though I have access to it) was that the test manager piece didn't support silver light 4 and our application was on that technology.
The latest update for Test Manager now has this support. So yes I am looking forward to using this to test.

Some pieces that I am really excited about
  1. Manual test cases - video recording. Really if I can have a witness to all my testing I would not have to waste my time battling, answering, writing clearer notes, etc. I would have it all in my recordings.
  2. Fast forwarding - I can fast forward my test cases. Super cool.
  3. Lab Management - If I can get the same test environment that takes 6-8 weeks to build in hours, I would be testing so much more than fighting paper work to get the lab machines.
  4. Use manual test cases to create automated test cases - really that just makes my life a lot easier. Automation and manual testing wont be in two different tool/technology. They will be based of each other.
  5. Dashboard - Requirements, test cases, execution, defects - when everything is in the same system, my metrics would mean so much more real time. I wont have to massage my data to co relate the different elements together.
I will be playing with this tool for the next several months. I will add more information as I learn more. But this is one technology I am excited about.


For more information visit
http://www.teamsystemcafe.net/Resources.aspx
http://msdn.microsoft.com/en-us/events/bb944781.aspx

Sunday, November 14, 2010

My Take: Weekend Testing Americas - Session 1

I was part of the first Weekend Testing American Session on Saturday Nov 13, 2010 from 2 - 4 pm CST. Michael Larsen was the session coordinator and he helped moderate the session and also presented the objective and mission.

You can find more information @ WTA01 - Let's Dance

There were 21 people from all over the world including India, Canada, Israel, etc. It was nice to see so many people from different locations, different experience, different testing ideas come together to try to work on one goal - making Weekend Testing American a success.

Mission was to test StepMania 4 (open source clone version of Dance Dance revolution) with a partner. We had to choose a partner within the first few minutes and then test it for an hour. We regrouped after that to discuss what went well and what didn't.

I learnt couple of things
  1. Don't forget the mission. I jumped into trying to test the application. The goal was to work with your partner.
  2. Testing is learning - Michale Bolton brought up a good point of how learning really is part of testing and not separate.
  3. Rapid Reporter - A cool tool to take test notes especially during exploratory testing. Thanks to Shmuel Gershon I now have a tool that I can use everyday.
  4. Skype has a share option where you can share screen. Another great tool that I will be using a lot. I used to use MSN Messenger sharing but it has its glitches and live meeting takes time to setup and is one way.
  5. There are some good testers out there who love to share knowledge and are good mentors. I met a very enthusiastic group during this weekend testing session.

Friday, November 12, 2010

My Take - TCQAA Meeting - Cloud Computing

Paul Selway from redpath talked about cloud computing last night at the TCQAA meeting. Very technical but informative presentation. He started with a high level picture of what cloud computing is and narrowed his focus to Iaas (Infrastructure as a Service). He used some good analogy to explain the concept of cloud computing. He compared the growth and evolution of cloud services similar to generating and distributing electricity.
He explained the cloud in real easy day to day terms. He asked us to stay away from cloud washing (like brain washing). He gave us good pointers on when to start thinking about the cloud and when to absolutely stay away from it.


I also had a good discussion with Barry Dietrich on how cloud will tie in well with lean and agile. We also talked about how we can take it back to our work plays to see how best to jump into the cloud.

Bonus for members who are still thinking about attending these meetings: There were several recruiters there looking for Test Leads and testers. Also there were couple of company representative's who were looking for people. The test market is opening and if you are looking for a job feel free to attend TCQAA meetings.

Thursday, November 4, 2010

Strength Accelerator - The Greatest Value you bring to the team - Coming Soon

I recently took the Strength Accelerator Test - its part of the new tool that Marcus Buckingham and his team is working on. Marcus Buckingham is the creator of the original strength finders assessment and has created several since then including one for women The Strong Life Test for Women)

I have taken the strengths finder and strengths finder for leaders. Here are my results for the top 5 in each category
My Top 5 Strengths
Significance
Communication
Strategic
Maximizer
Woo

My Top 5 Leader Strengths
Activator
Communication
Strategic
Woo
Futuristic

I took the assessment and was curious to see what else can they tell me about me after already learning so much from the previous two tests. Guess what I learned a lot. The results talk about "Strength Zones". It gives you the top two strengths that you bring to your team. My top was creator and I was surprised and curious till I read the description - "You make sense of the world, pulling it apart, seeing a better configuration, and creating it. 

The second strength zone was pioneer - You see the world as a friendly place where, around every corner, good things will happen. Your distinctive power starts with your optimism in the face of uncertainty. 

Together they make a lot of sense and I feel like it will help me channel my focus on using my strengths to do at work and home life. The results (20 pages long) has suggestions on how to use my strengths and how to get more value out of the two zones together. I will have to read the results several times and try to understand it all.
Over all a very good test and I would recommend it to anyone who is looking to learn about their strengths and also to find out how to use the strengths in their everyday life.

Wednesday, November 3, 2010

Friday, October 29, 2010

Review - ALM Expo 2010 - Online conference for ALM

I recently attended this online conference (Oct 26-28, 2010). It was a three day conference on ALM, Cloud and Agile. To know more about the conference or view their pre-recorded session visit
http://www.cmcrossroads.com/alm-expo

Day 1 was focused on ALM. Absolutely great for people like me who know where little about ALM or what the new trends are. Cloud is here to stay and Application Life cycle Management is a key element to be successful in the cloud. You can go back and listen to the sessions. The Keynote speaker and then the discussion that followed were really informative.

Day 2 focused on Agile (my favorite part of the whole conference). I got a second chance to listen to Jeffrey Fredrick. Again the keynote discussion was good. Had some good polls during the sessions.

Day 3 focused on Cloud. These sessions tied the other two days together. It helped put the three (Agile, ALM and cloud) in perspective and how they are interconnected.

They had a scavenger hunt all three days for some reason I could not find the questions as per the instructions. I wish there was someone to help with that.

Wednesday, October 20, 2010

My Take: Agile Comes to You (Minneapolis Seminar)

I was at the Agile Comes to You conference on Oct 19th 2010. It was a good place to learn about agile if you have not heard about it yet. A place to learn more if you are just learning about agile or are in the initial stages of implementing agile. Its also a good place for people who are already into agile and just want tips and tricks. The session was from 9 am to about 2:30 pm and included breakfast and lunch.

The keynote speaker David Hussman from DevJam was really good. His dude's law on value/how/why was simple thinking. He could as well call it common sense law or people's law and really it applies to IT or software development in general. You don't have to be agile to follow it. He focused on test driven development and talked about how agile and test driven development go hand in hand. His theory on simple-complicated-complex was really interesting. If you do get a chance to listen to him, please do so. You can find more about him at DevJam.

Michael Johnson from Make Music Inc spoke about how his company adopted agile successfully. He talked about how not everyone was into agile but once they saw the value it was easy to get them all to be as excited as others.

James Talbott from AccuRev talked about Agile Workflow Economic Management - his focus was on what is the outcome instead of the how or efficiency. His analogy using football was really good - end of the day its the score not the yardage that counts. So true.

Jeffrey Fredrick from AnthillPro talked about The Co-Evolution of Continuous Integration and Agile. His checklist manifesto was really interesting. He talked about having simple processes in complex environments. He talked about how people matter more than anything.

There were product demos in the end. It was nice to see some tools that support Agile: Rally, AccuRev and AnthillPro.

Sunday, October 17, 2010

My Take: TCQAA Meeting and presentation: Speaker Alex Dietz

Twin City Quality Assurance Association is a local non-profit organization and is the networking place for software testers in the Twin Cities (Minnesota). To know more about the organization visit TCQAA website. I and my colleague recently decided to drop in at their monthly meeting. I looked up their website to see the presentation topic.

Speaker: Alex Dietz
Topic: Principles of Software Verification and Validation for Medical Imaging.

I went to the meeting thinking there is nothing I learn about Medical Imaging that I would be able to use for testing in the financial industry. I am glad I was proved wrong. I went in to learn about an industry that I was curious about and have never been involved in. The speaker was very engaging, humorous and lively. Though his focus was on testing software-heavy medical imaging system, his approach for verification and validation can be applied to any industry. He talked about risk based testing and involving end users (doctors or medical specialist) to test. He also focused on reducing errors that are caused due to human errors to compensate for errors caused during the creation of the image.
I learnt a new word too: phantom. No, not the comic we read years ago. These are medical images that are used for testing. Look them up and you will find a lot of interesting information.

There were some good discussion in the end during the Q&A on the trends and problems medical industry faces. He talked about issues related to storing millions of images, sharing information between hospitals and doctors and network issues that this industry depends on for storing and sharing information.

You can find the abstract of his presentation at TCQAA events page.
_______________________________________________________________

Few other benefits:
If you are looking to network or are looking for job openings, TCQAA meetings are a good place to start. There were several recruiters there at the meeting who talked about Software testing jobs in and around the twin cities.

I also met an ex-colleague of mine after several years. This is also a place where you meet other software testers and can learn/share information.

Who can say no to coffee and cookies right. So do drop in when you can to attend their meeting.

Thursday, October 14, 2010

Cloud Computing 101

For weeks now I receive at least one email everyday mentioning cloud computing. I was curious but not motivated to find out more. Finally seeing a flurry of tweets I decided to look this up. I went into it thinking its a product that is being marketed hard and fast.

I was so wrong (ya a true duh moment) and not alone in the "I don't know what cloud computing is?".  VerizonOne survey IT professional in 2009 and revealed that 41% of senior IT professionals admitted they didn't know about cloud computing. I am sure this number has decreased since then. Compute world predicts this as one of the trends with the most promise in 2011.

What is cloud computing?
Cloud computing from what I understand is Internet based computing technology that uses Internet and remote servers for data and applications. Users (consumers and businesses) can use applications from any computer through the Internet on demand. This is the new buzz and is being forecast as the next big thing in the technology industry. Cloud gives the opportunity for collaboration without boundaries. Companies can increase capacity without investing a lot in infrastructure. More and more companies will jump into this bandwagon and will promote integration of services across borders. Services can be offered as software (SaaS - software as service), platform (PaaS - platform as service) or as infrastructure (IaaS - Infrastructure as a service).
 
Who is in the cloud?
Google
Microsoft
Hewlett Packard
IBM
Salesforce
Amazon

Why it works?
  • User really does not have to know the technology behind the services
  • Can be used by anyone over the Internet from anywhere
  • Low cost when compared to owning the infrastructure in house
  • Maintenance is easier since its not installed locally in each user's computer.
What are some concerns?
  • Dependency on the Internet
  • Loss of privacy (providers can monitor or control the services)
  • Compliance to regulations (more expenses involved when trying to be compliant with regulations)
  • Security when its being managed over the network or outside infrastructure
Where can I get more information?
Bird-watching in the cloud
What is cloud computing?
InfoWorld
IBM

(Please note this is based on what I learnt in the last few days out of curisoity. This is not everything about cloud ... just a short intro.)

Monday, October 11, 2010

IT Knowledge Exchange

A great site for getting answers to your technical and non- technical questions.
http://itknowledgeexchange.techtarget.com

You can ask questions or answer questions. Its for both learning and sharing knowledge.

Thursday, October 7, 2010

Its Complicated!

Its complicated! Nope not the movie but everything else I guess.

I was talking to a lady at the bus stop this morning about how even simple things like the cell phone is now complicated. How some people over-complicate even the simplest processes and how we like to add arms and legs to something that probably is like a one cell organism or simpler.

Is it the curse of the new technology? Is it the expectations that people have from things? Does complexity add value?

Stanton Champion recently wrote We Really, Really Like Feature Bloat (UTest) where he talks about how we want the coolest features over simplicity. When a customer is out there looking to buy a computer, laptop, phone, Internet connection, cable TV, phone services, they really are looking for what all can they get for the money they pay. What are the coolest features out there, what is the latest model, will I get more if I spend a more.

Technology is getting outdated at a faster rate than it used to a decade or so ago. Couple of years ago when you buy a phone, computer or even take television, you would think of long term use of that product, something that lasted forever (well almost forever). But today its not about long term, its about what all is available (features, uses, how latest is the technology) and how fast or cool it is. People buy it with the notion of replacing it a few years or even months down the line.

Don Norman talks about this in his article on Simplicity is Highly Overrated. He talks about how people want more (complex) features for more money. If we look at this closer it does make a lot of sense. Customers pay more when they think they are getting more: more as in more features, complex features, latest technology, more settings, more applications, etc.

Or is complexity really a perception. Do some people really find latest technology to be less complicated than others? My 10 year old son knows more about my cell phone than I do. I go to him for help with settings or to learn about the new features and applications. So for one its real simple and to the other its real complicated. Who defines complexity? Does the 80-20 rule apply here? If 80% of people use a feature then that feature is simple. If less than 20% people use a feature does it mean its harder or more complex?

I think complexity can be attributed to perception to some extend. Once my son shows me how to use a feature in my phone and once I start using it I am more comfortable with it. My perception of how complex it used to be changes. Does familiarity breed simplicity?

Joel Spolsky on the other end talks about how simplicity actually complicates product development and creates ambiguity. People want simple but with everything else along with it. When designing a product we like to think we are making a simple and easy solution, then we add to it all the features that we think the customers want, then add everything that the competition has, then add something extra to make it cool or stand out. What do we have here? I don't think its simple for sure.

What does all this mean to software development and marketing? Well it means managing perception and adding value so we have happy customers. Happy customers make happy companies and happy employees.

Friday, October 1, 2010

Web-Based Applications - Emerging Trends and Challenges for Testing

With the rise of Internet, there is a rise of web applications. Applications that were previously installed in personal computers or specialized labs are now accessible through the Internet. With this emerging trend, testers also have to keep up with the changes and learn to adapt to work with the challenges of web application testing. The end result of testing for traditional application or web application may be similar (the application meets the requirements, has no critical bugs and meets the business and customer needs) but the means to get there may not be the same. The traditional testing methodologies have to be modified and adapted to the new technological demands.
Some reasons as to why web application are more challenging
  1. More concurrent users than traditional client servers or standalone (load testing and performance comes into play)
  2. Users spread all around the world (cannot test from every country or even states in US)
  3. Users may have a different platform or environment that they are using to access the web application (may not be possible to test all the permutations and combinations)
  4. More components and technologies are involved when developing these applications, third party tools, COTS or in house built from scratch tools (more things happening in the back end that are not always in control when testing these environments)
  5. Less control over the test environment (its no long something that is installed on my computer where I can control most things happening in the environment)
What should testers do?
  1. Keep up with technology, learn as much as they can about the technology that is being adopted for their applications be it third party tools or something that is built in house from scratch.
  2. Be ready to adapt to the technology - traditional methods may not work as successfully and testers have to be open to work with technology.
  3. Talk to developers about technology and try to understand the back end/UI/Network, etc as best as possible.
  4. Focus on the weaknesses of web applications and try to capture issues in those areas as early as possible e.g. security, network, connectivity, firewalls, etc.
Steps for testing web applications
  1. Understand the design - its important to understand all the components involved here like where is the application hosted, how is it connected, is it thin or thick client, are any of the components installed in the local machines, etc. Understanding how the components work with each other will also help with planning.
  2. Test Planning - Document the plan well and involve developers, architects and security team so that the appropriate risks can be documented and discussed well in advance. Also planning for test environments that mirror production is important. Talk to customers if necessary to see if there are additional network firewalls or settings that you would need to work with during testing.
  3. Testing - Test as close to how the customers would be using the system. Get customers involved if its possible or else plan for user acceptance testing. Define the level of testing ahead of time and get your team involved in this.
Testing web application is challenging and test leads have to be well prepared to face technology and its challenges.

Tuesday, September 7, 2010

Book review: Software Testing and Continuous Quality Improvement - William E. Lewis

Book review: Software Testing and Continuous Quality Improvement (Third Edition) - William E. Lewis

I recently finished reading this book. I am really impressed with the third edition. This is a good refresher for anyone involved with software development be a tester, a manager or a project team member. This is a good book for a company starting a new testing team or updating their processes.

We recently started using agile development process at work. This book really helped me define and update my test plan, risk ranking and test metrics to work with the agile framework. This book explains testing in Service Orientated Architecture (SOA) environment and the building blocks of a Testing Center of Excellence (COE).

There is also a section on free software tools and a checklist to pick the best tool that fits your organization. The book includes a CD with templates for documents discussed in the chapters.

Tuesday, August 24, 2010

Power (or not) Point - Presentation

I recently read James Christie's The medium’s PowerPoint, what’s the message?. It was one of those articles that makes you stop doing what ever it is you are working on and start thinking.

I will be the first one to admit that when I started using PowerPoint I thought it was the next best thing after Windows. I fell in love with the application. I used it everywhere and anywhere I could to do presentations. The cool animations, the control that I had on how to share the information, the colors I could play with - well the list just goes on and on for things I loved. Yes I loved the power of PowerPoint. I still like the power but the big question is how much power do I really have if I don't cater to my audience.

Presentation is not about the presenter, it is about the audience and like James says "the message".

Yes power point is still a powerful medium but as presenters we have to find the right balance. Depending on the audience and message the presenter has to modify the presentation. Using power point is a choice and should not be considered as the most important element.

Start with "the message", understand "the audience" and "the medium" will automatically fall into place.

Thursday, August 12, 2010

Defect Density - How and why I gather this for my test metrics

When I was asked to calculate defect density for my projects, I wasn't quite sure what it was. I researched it online and found several articles on it. Most of them recommended using lines of code to measure defect density while others suggested using function points.

Defect density is usually calculated per thousand lines of code. As a tester I was struggling with this as I didn't have access to the codes and even if I did get access, I couldn't figure out how I would related the defects found to the code line.

I had to be more creative and decided to try calculating this by builds being tested. I had metrics for defects found by build and test cases executed by build. I also knew the features and changes being delivered in each build and this would help me narrow down areas that testing could focus on. 
So the formal I used to calculate defect density was
Defect Density = Number of defects found per build/number of test cases executed per build

I calculated this for each build and also tracked cumulative metrics to look for trend. Here is a sample of one of the defect density reports I created

Why calculate defect density?
Here are some things that this test metric can help with
  • Identifies cluster of issues in an area or feature to narrow down root causes
  • Identifies areas that need additional testing focus, re-engineering or redesigning
  • Identifying defect dense areas help focus limited resources in those high concentration features
  • Measures amount of rework - identifies features or areas where rework needs to occur or has occurred
  • Compares relative number of defects in different components of the software application
  • Predicts remaining defects when compared to expected defect density based on the trend
  • Determines if testing is sufficient
  • Creates baselines for standard defect density
  • Can be used to measure quality improvement activities for later releases
This is not a perfect method to collect defect density. There are some drawbacks to this method of collecting this metrics
  • Not all test cases will be similar. Some may be longer and some smaller. There could be more than one defect associated with each test case or multiple test cases could fail because of one defect. These have to be taken into account when the data is being looked at.
  • Some defects may not be related to the test cases at all. And its hard to decide whether to count defects that are found during the execution of the test cases or not.

Tuesday, August 3, 2010

My Take on CSTE

Yes I am certified. There was a time I said I will never take this test. I put it off for about 4 years. Then finally gave in and said why not get certification in software testing since I don't plan on leaving this industry anytime soon. I mean no harm in trying something at least once. I started preparing in the summer of '09 and decided to write at the end of the year. I wrote the exam Nov 09 (right after thanksgiving) and got my results Feb 2010.

Some things that helped me
  1. Group studies - bunch of us from work decided to study together and it helped a lot. It was easier to discuss and understand the content (software testing and testing management as defined in the book) instead of sitting alone and trying to figure it on your own.
  2. Index Cards - I wrote almost everything important down on index cards. This was almost everything in the book. Sorted them by chapter and would have them with me at all times. I would go over them when I got a few minutes.  Then every week I would sort them into separate bundles - easy, hard, not sure. Pretty close to the exam the easy bundle was small and hard was small but the not sure pile sure was high.
  3. CBOK Made Easy - We think we know testing but the definitions and explanation here in the book are really based on testing as we knew in 80s, 90s and some from the current decade. But remember to pass in this exam you really have to write their definition and also choose multiple choice answers based of the content of the book. We may or may not agree to it but then we are not being tested for agreement, the test is all about how much you know about testing in terms of the CBOK. Found a link to a book which made the CBOK easy. If you google you can find several different versions of this book. It basically defines the CBOK and also makes it easy to understand. I read both the books in parallel and it helped a great deal. When you start studying, you really have to unlearn what you know and learn their definitions at least for the test.
  4. My strategy
    • Start with chapter 2. If you start with 1 you will loose the motivation to continue. Chapter 1 is the largest, hardest and confusing chapter. So start from 2 and go to 10 then come back to 1.
    • Read all chapters at least twice. First round take your own time, write the index cards. Second time do it faster and also use index cards to refresh some definitions, etc.
    • The days before the exam only use your notes and index card and go back to the book if you need some examples or are looking for something specific.
    • The days before get rid of everything. Get good nights sleep and leave the rest to what you already know. No use trying to get more confused.
  5. Actual exam
    • For essay questions - write the key words. Don't waste time explaining it all. The computer doesn't care.
    • For multiple choices - try to eliminate as many answers as you can and work on the ones that you think fit the most.
Now that the I have the certificate what did I really learn?
  1. CSTE is a good refresher - It really is a refresher. There are things you forget over time and studying for this exam gives you an opportunity to refresh your knowledge. 
  2. Common language for co-workers - A lot of times we get into discussions with co-workers and having this common language from one book helps reach conclusions. We just grab the latest version of the CBOK and look up topics that we want to discuss.
  3. Similar foundation - This really gives a the team a similar foundation. People come from different background, different education level, different experiences, so this really ties them all into one category (good or bad).
  4. Substantiate your knowledge - It sometimes help to read about software testing and testing management to substantiate what you already know.
  5. Certification - Its now an addition to my resume, my profile, my experience. No one can take it from me.
  6. Learn something new - There are things about testing that we either didn't know or knew vaguely. This book really gives us information in various areas that we may not have experience with example I have never worked on mobile application or done security testing. Reading those chapters really gave me some new knowledge.
  7. Recertification - for recertification you either take the exam again every 3 years or you read/attend seminars, etc. I think this is good a motivation for education, attending seminars, learning something new etc. I don't think I want to write the test again so I will be focusing on the latter which adds more value and experience than retaking the test.

Book Review: The Buck Stops with You! - John Graci

When Leaders Lead, Employees Become Motivated

Read this book over the weekend. This book has a lot of common sense strategies to help managers motivate their employees. John Graci talks about how managers can really make their employee feel good and want to come to work.

A few of his techniques that really made me think and want to apply:

Listen up - really this shouldn't just be managers we should all be listening to each other at work be it a team member in your project team or a co-worker who may have a suggestion.

Perception is reality - no matter how much we try to deny or justify if someone perceives something to be true then its part of their reality and its hard to change that. If someone things you are rude in your emails even if you don't think so, then its true for them and we cant change that. I think perception management is important for any manager or employee if they want to succeed.

A few other things he talked about were
Involve employees in change
Coach don't lecture
Give recognition
Delegate Authority

Sunday, August 1, 2010

Tracking Test Metrics

Tracking test metrics is really an important and integral part of a test lead's job. It helps track testing progress, testing trend and helps with telling a story with data. The data is always there. The leads responsibility really lies in determining what to collect and how to present it so as to tell the testing story to the stakeholders, management, project team, etc.

What to collect?
This question is not as complicated as it looks. It boils down to 3 things

Progress:
What did you plan for the release and where you are in a weekly basis. This is based on two things: Test case creation and test execution. Number of test cases planned versus number of actual test cases created. Number of test cases planned to be executed and number of actual test cases executed with their status (pass/fail).

Some examples of metrics for gathering Progress are
  • Planned Test Cases Created versus Actual Test Cases Created
  • Planned Test Cases Executed versus Actual Test Cases Executed
Trend:
Trend is more along the lines of the direction testing is taking and what you can predict from the metrics. What does the weekly execution data tell you as a lead? Do you see more failures for a new feature or an existing feature? Do you see more test case execution (productivity) occurring when you get a build early in the week than when you get a build mid week? Its all about letting the data talk to you. At times you wont even see a trend till you see the same data with different context.

Some examples of metrics for gathering Trend are
  • Weekly Test Cases Executed
  • Weekly Test Cases Created
  • Cumulative Defect Density 
  • Weekly Defect density
  • Open and Closed Defects Per Week
  • Defect Age
Quality:
This boils down to the defects that testing team finds. Defect data can be sliced and diced in several different ways and each time it really talks about the quality of the product. Based on the data that is gathered, a lot can be said about the requirements, code, design, product, test cases, testing process, customer, etc. Analysis of defects can really tell the organization a lot about how well we are doing our job and also expose our weakness that can then be easily rectified before the product goes out the door. Finding defects is in no way a negative to any one team or department. Its just a way to judge how we are doing.

Some examples of metrics for gathering Quality are
  • New Defects found per week
  • Defects closed per week
  • Defects found per feature
  • Defects found per build
  • Defects found by Severity
  • Defects found by Priority


Saturday, July 31, 2010

Fan - Marcus Buckingham

Yes I love the term Facebook uses. I mean really Fan I cannot find a better word for it. So here goes this section is for people, places, things that have inspired me over the years. And I am starting this with Marcus Buckingham, a speaker, an author and a lot more.

Marcus Buckingham has inspired me to look at my strengths. Yes its not easy for me especially having grown up in India where the focus is over all growth and if there is a subject that we are not good in then the whole family and community focus on that one weakness instead of the million other strength we have.

Going back to Marcus, his books and seminar have inspired me a lot. I feel more confident in using my strengths and also have learned to deal with my weakness.

I was lucky to have been able to hear him in 2009 in Alexandria, Minnesota. Here is a picture I took with him.

*Yes he is as cute face to face as he is in pictures.

Friday, July 30, 2010

Seminar: Presenting Data and Information - Edward Tufte

Recently I had a chance to attend Edward Tufte's seminar in Minneapolis, Minnesota. From what I hear Mr Tufte presents this couple of times a week. The seminar was held in the Minneapolis Marriott City Center. This was the first time for me at this hotel. The actual seminar was from 10 am to 4 pm. You could go in early to gather your package and also have some one one one time with Mr. Tufte. He even signs your books.

So I went in, got my package and agenda. There were some recommended readings before the seminar started. I was pretty impressed with the check in process. Show the bar codes, get checkin and gather the package. How easy.

Well I gather my stuff and sit down to read the books. It was a little hard to focus on the readings with trying to keep an eye on whether Mr. Tufte will come to you to sign the books since he was walking around and signing them at random. The package had 4 books written by Edward Tufte.

The Visual Display of Quantitative Information
Envisioning Information
Visual Explanation
Beautiful Evidence

Anyways I was lucky enough that he stopped by and talked to me for a couple of minutes and signed my books. We got into a discussion about presenting data and audience. Without going into the actual details I will give you a gist of what I learned from him in those 10 minutes that helped me understand the rest of the presentation.

Its all about the content. Know your content.
Be brief but precise and intense.
Do "what ever it takes" to present data.
Get a good template and stick to it.
Use super graphics.
Simple.

I really enjoyed the presentation though it takes a while before you really connect the dots. I don't necessarily agree with everything he had to say but all in all I took several pages of notes and think that I will be applying a few when presenting data in the future.

I will probably talk more about his seminar in my future blogs Also the 4 books we got totally worth attending the seminar for.

Metrics - A Tester's True Friend, Not A Foe

Metrics Oh Metrics it took me a while to understand you. Now that I know you I appreciate your help with presenting status or talking to the project team.

Yes its true I love metrics and I will give you ample reasons so you would love them too. Well maybe not love but atleast start appreciating them better.

  1. Metrics helps tell stories with data. "Story" lets start with defining it. Story really is what you are trying to achieve example a new release for your product, hot fix, user acceptance testing for a product that interfaces with your application, or a mini regression cycle for production release. So the goal of the metric here is to tell your story - where you are, where you should be and how much is left.
    • Where are you? Really that is your actual work say how many test cases did you write so far, how many test cases did you execute and what is the status of those tests.
    • Where should you be? - say today is the last day of your test case creation phase you should be at 100% test case creation complete but if you are in 90% then your data should show where you are and you should be able to explain why you are not at 100%.
    • How much is left? - this is very important information - what is left to do, how many test cases do we have to execute or how many defects have to be retested. Imagine if your project manager comes to you and says "Sorry to say but we have release one week early and since testing team will be affected the most, can you tell me how many more resources would you need to complete your release?" You should be able to pull these numbers from your metrics that you have been maintaining. You should be able to tell exactly how many people you need to get this done in the time frame you have.
  2. Metrics helps predict trend. Trend is important. It helps replace words like my gut feeling says this is not a good build or my gut feeling says we have a lot of defects. No your data should tell you if you have higher than unusual defect density or if a certain build has more defects for a certain feature that in turns shows the high risk area that needs regression testing. Really let the data support your gut feeling. Next time you have a one on one with you manager or a team meeting talk with numbers and you will see a huge difference in how they perceive your gut feeling when you tell them the product is not ready for release.  
  3. Numbers or data have the same meaning. It cannot be misinterpreted. If you have executed 100 tests and 50 fail then your failure rate is 50% there is no two meanings to it. Well ya you can say 50% pass rate. Either way it means the same. So when we bring numbers to the table, there are fewer arguments over what the data means.
  4. Metrics shows quality. Metrics really shows the quality of the product and tracks the quality too. How many defects were logged, how many of those defects were fixed, how many are critical? For test exection it shows how many test have failed and how many test cases have yet to be executed. Metrics speaks volume for the quality of your work, your teams work and shows the readiness for the release.
Takeaway - If you create good metrics it will help you make your life easy. It helps you manage your testing project better and also get the rest of the product team off your back when they worry about status or progress.

Thursday, July 29, 2010

Why Testing?

Over the past several years people and interviewer's have asked me this question a lot. Why are you in testing? Or why did you choose testing? And I have always replied with "Why not testing"? I didn't grow up saying I want to be in testing, I didn't think about it when I was in college. I was offered an opportunity long ago by a friend. He said try it out and don't do it if you don't like it. So I gave it a try and never looked back.

I believe there are two sets of people. People who understand testing and people who don't understand testing. No I don't mean definition wise or theoretically. I mean more in terms of concept. You can learn as much as you want about testing and claim to apply things you have learned. The real understanding comes from how you handle it in real time.

Lets take for example the definition as stated in the CSTE CBOK
"Testing is the process of evaluating a deliverable with the intent of finding errors."

This sums up the testing process. The greater problem lies in understanding errors, how to find them, and also what deliverable we are talking about. Its really the thinking behind the actual job that counts. You can dive in deep and test or you can float in the job of trying to test to come out on the other side really not knowing what you have to do.

Testing is an art and a science. And you have to understand both to really understand testing.
Definition of Art (Merriam-Webster)
  • skill acquired by experience, study, or observation
  • the conscious use of skill and creative imagination especially in the production of aesthetic objects; also : works so produced
It really drives down to the fact that testers have to be open to a lot, should be willing to learn, be a good observer and willing to learn from experience. These things that we learn over time are skills that have to be applied to the job to be a successful tester. It is very important to use your skills and imagination. There are no limits or boundaries to testing. One does not stop at just testing, testing involves learning the system, putting on different thinking hats (Business Analyst, Developers, customers, end users, support line, etc). Its about being able to switch gears and roles to really bring the best of the product forward and identifying the risk (weaknesses) to help our organization to mitigate these risks. Its about the big picture - release of the product to the customer and to meet their requirements with as few risks as possible.

Definition of Science (Merriam-Webster)
  • the state of knowing : knowledge as distinguished from ignorance or misunderstanding
  • a : a department of systematized knowledge as an object of study b : something (as a sport or technique) that may be studied or learned like systematized knowledge
  • knowledge or a system of knowledge covering general truths or the operation of general laws especially as obtained and tested through scientific method b : such knowledge or such a system of knowledge concerned with the physical world and its phenomena
There are a number of processes and functions that can be used for testing. The goal that the testers are trying to achieve are finding as many risks (defects or bugs) before the product goes to production. There are a large number of scientific processes that testers can apply to get this done. CSTE CBOK get into a lot of them in detail and I don't want to get into the actual process here.
 
The point I am trying to get to is between being an art and science, Testing often the most misunderstood job in the technology industry. Its not about breaking anything. No its not breaking software. Testers simply bring out the risk (defect, bugs, weakness - each one defines it differently ) of the software. And this job is no less no more than any other technology job that is out there.
 
So why testing? Because its art and science and needs skills. I have the skills and am a successful tester. So any tester who still doubts their role please don't.

Why "Today's Big Picture"?

I have been meaning to start a blog for a long time. I debated on the topics, the name and the purpose. In the end I decided I just have to start it and depending on the day I could bring what is important to me to the table here in my blog. As a full time working mom, when I have a lot of things going on, I always remind myself that the most important piece is the big picture. Technically big picture is my goal. Sometimes its as simple as get a document out before end of the day or it can be a month long project where I need my team to help. When a lot goes on I try to break things down into smaller chunks and this helps with positively marching towards my big picture.