Crazy 2am iPhone Order – Would the site work?

I stayed up until 2am to order my iPhone.  Crazy, maybe… but I’ve held out for an upgrade from my Samsung windows mobile device for over 3 years.   I call my current phone “the brick”.    It was an amazing smartphone, long, long ago.   I’ve been fortunate to have the latest devices through work, so I have been patient waiting for Verizon and Apple to provide a solution.

So… it’s 1:50 am and I anxiously check the countdown page.   They are ready for me to order.   I am worried for them. Would the site hold up to the load?   Did they test all of the order processing workflows?   What about error conditions and unexpected user data?   What about my user data?

The first attempt went as I expected.   It failed.   I upgraded 2 different phone lines, carefully selecting my options and just prior to checking out I get an “error – we are trying to correct as quickly as possible… try again”.   Argh!

I immediately wonder what test scenarios would have captured the failure. Could this have been uncovered with additional load testing?   Was it the fact that the landing page was updated prematurely and the rest of the order pages were not in place?   Was it that I was slow the first time through as I read each of the options and reviewed help?   Was it an issue with the help pop-ups and a focus issues once I returned to the main order workflow?

Then I wonder how many orders were lost along with mine.   What was the business impact?   Maybe none considering the time of day and the insanity of those of us who would try over & over until the order was processed.

I have to start over.   This time, I make it through with no issues.   I am careful to capture my order information in case there are issues with the backend systems.   My confirmation email is delivered about 30 minutes later.   I am relieved, I was concerned I would never get an email.   I quickly check the site to confirm my order is in the system.   Indeed it is and the screen indicates that the order is being processed.   Then I check the site to confirm that the charges and order details are correct.   Everything looks right.  Excellent!

Finally — the waiting is over.   Well, not really.   Now, I keep checking to see if my “order has been processed” and my “phone has shipped”.

Let me check now… yes I checked just 5 minutes ago.
YES!  “Your order has been shipped.”    Now, I will keep checking to see if the courier and tracking number is posted and of course I will listen intently for the doorbell.  🙂

Agile for the Lazy

Are agile methods a divine gift for the lazy?  Have you run across folks who use the adoption of Agile methods as an excuse to get out of work?  

I was talking with a former boss and one of my favorite mentors yesterday about fast moving development teams that adopt Agile methods.    We were discussing the examples of how adopting agile methods can truly help teams improve throughput, quality and communication.   There are some great success stories out there.  Then we talked about the common excuses.

I like to think of it as Agile for the Lazy.   You have worked with them and spotted them on your teams.   These are the folks are often well-intentioned, but are simply looking for an easy way out.   Agile is like an amazing gift to them.   Finally, a reason not to do the hard stuff!

Software development is hard.  It’s detailed, tedious, requires extreme attention to detail (by someone) and requires rigorous organization to produce quality products.    We’re all looking for better ways and methods.   Agile can be perfect for folks wanting to find a quick exit.

In talking with colleagues, it appears that one of the most famous lines is “Agile is great, we don’t have to document anything anymore!”  Here are some other lines you may have encountered:

• “I told them everything they needed to know in the stand-up meeting.”
• “Now that we’ve started Test-Driven-Development, there’s no other testing that we need to do.” 
• “Why do we need a project plan?”
• “Well, we’re agile, why can’t you get the project/release out faster?”
• “The test and doc team just has to do a better job of keeping up.”
• “Now that we’re agile, we can add a lot more features/changes.”

So, how do you handle the lazy?  First, let’s go back to the basics.   If you are adopting Agile methods you need to know where you are going and you need a plan to get there.   You will need a vision and you need to recognize that you have to re-engineer your internal processes for the Agile methods you are adopting.   The overall plan should change as you learn, which is the whole point.   

What about implementing change?   Changing your processes means you need to look at your entire development life cycle as changing.   You need to re-engineer to make sure that the Agile methods fit and work for the team, that tool changes or work flows are considered and communication methods are defined.  The team will need training to understand how the changes will affect them.  The team also needs supporting objectives and performance targets that will support the changes.    You need to also choose how quickly process change can occur.  What can the team absorb without derailing your development schedules.   If things don’t work well in your environment, then you change.  That’s the whole point.

Worst thing you can do is allow the lazy to control Agile adoption.    You may end up in a situation where the entire process becomes dysfunctional and the team can no longer execute effectively because each person has their own interpretation of the vision and goals.

Here’s some excellent related posts that delve into this same topic:

From Kevin Schlabach:   http://agile-commentary.blogspot.com/2008/10/agile-practices-as-gateway-drug.html

From Mike Cottmeyer  :http://www.leadingagile.com/2008/10/agile-or-iterative-and-incremental.html

What about dealing with management who thinks Agile just means more projects/applications/features, faster with the same resources?   Now that’s a topic for another day.

Discovering Change or Defects?

What are your goals for testing?  How is your test suite positioned with your team and management?  What do you think you are discovering – Changes or Defects?  

I believe these distinctions are not always clear to a Development or Test team and definitely are not often understood by management.   Folks in the trenches typically know the difference, but may not realize everyone else is in the dark.   A test suite, automated or manual depending on when it’s run can detect changes in an application or defects.  

The difference is that in an iterative/agile environment, the team is probably not writing down all or possibly any of the planned changes.  Changes usually are coming at the test team at break-neck speed.  So, the test suite is probably not keeping pace with the changes.  Therefore, when the suite is run, you are detecting changes.   The change may be expected/planned or it may represent a design issue or defect.   Some may be obvious defects.  Aside from the obvious, the folks running the test suite are then required to figure out if the issue is with the test (automated, if applicable) or if the change is expected and the test needs to be revised.   There’s nothing wrong with this approach if this is what you intend to do and you are not expecting to be running test suite for the purposes of detecting defects and validating requirements.

So, why does any of this matter?  Let’s go back to goals.  If the goals are to have an up-to-date test suite running where all planned changes are accounted for and you are discovering defects, there’s a huge mismatch if you are in a high-change situation.   The team may be expected to report many defects, however their time is consumed with trying to analyze the test failures — or rather the changes.

Ultimately, someone is heading for a discussion about expectations and the mismatch on what is being done vs. what was expected.    The team responsible for testing is probably complaining about the the situation and their inability to keep up, instead of understanding and highlighting the value they bring given the circumstances.  

So, what is the value?  If the development life cycle is fast and testing happens in parallel, then detecting change can actually be beneficial to the development team.   Okay, so perhaps the team could be better about identifying changes and providing that info to downstream teams.   If that’s not possible, the work is just pushed downstream.   Changes are identified in testing cycles, the change can be documented at this point and discussions have to occur to move forward.  Test cases can now be updated for the change and you move forward.   Could this have happened up-front?  Yes.  But, the whole point of this post is to understand the situation, understand how to best leverage the situation to help the team be successful.  

Now, the final question in the high-change environment, how do you create a test suite that can detect enough changes if you don’t have enough info to develop coverage for the changes?