I'm thinking of a number...

[textile]
During CAST my wife was forced to spend time with various people in our industry who spend too much time thinking about and talking about testing. She's a teacher, so she's use to dealing with kids.... I mean testers....

One night at dinner, she shared a game she played at work with a class of her students. It's a way for them to kill time waiting for other classes to finish so they can switch rooms. The teacher thinks of a number between 1 and 100 and the students try to guess that number in the fewest number of questions. Amazingly, the word on the street is that one student did it in four. That's the record.

Of course, hearing of this game, several of us immediately turn this into a testing problem. The game now becomes, "Identify the number in the fewest number of yes/no questions, where each time you guess a specific number it not only counts as a question but if you are incorrect, it counts as a false bug report." That is, if you ask, "Is the number six?" and are wrong, you lose street credit with the developers.

There were several interesting tactics applied. Let me share a few....

1) Our first contestant was Jon Bach (tester). I had the number. Jon asked a series of questions based on context. One particularly genius question was, "Is the number 42?" He asked this because earlier in the day we had been talking about The Hitchhiker's Guide to the Galaxy. Jon took notes on his tests and used a matrix of numbers (which became a standard practice with all contestants except my wife) to track his progress. He ran into a rough spot where he forgot to record the results of his third test (faulty lab procedures), "Is the first digit bigger then the second?" This cost him dearly in both time and street credit. His final count was 10 questions with two false bug reports. He was man enough to admit his mistaken recording and promised that on future tests it would not happen again.

2) Our next contestant was Julian Harty (tester). I had the number. Julian had the advantage of watching Jon ahead of time. Julian used a straight binary search and zeroed in on the number in seven questions, zero bug reports. His process was repeatable and efficient. No points for creativity, but clearly showed he understood the project and that he could develop a simple systematic solution. He continued to think about the problem for quite some time, trying to develop an algorithm better then a binary search, but in the end, he felt it would remain the best way to approach the problem in the future.

3) Our third contestant was Amy Kelly (teacher). I had the number. Amy has seen the pro's in action (so there's no mistake, the pro's are her students, not the testers). Amy used some simple binary questions to eliminate a large portion of the numbers up front. Then, with a limited number of solutions, she switched over to the brain engaged Jon Bach approach. Her best question was "Does the number have any curves in it?" This shows that she had not limited her analysis to the numeric representation that the others had up to this point but was thinking on multiple levels about the problem. Towards the end of the process, she did take a guess, which cost her. Amy solved it in nine, with 1 false bug reports.

4) I was the fifth contestant (tester). Jon Bach had the number. I had seen all three of them attempt to solve the problem and I had been most inspired by Amy's approach towards the end. Some of my questions included, "If you convert the first digit to a roman numeral, does it contain a V?", "If I subtract the first digit from the second digit, is the result negative?", "If you add all the digits of the number together, is the sum greater then 10?", and "If you divide the second digit by the first digit, is the remainder zero?" When analyzing my results, Paul Holland and I discovered that I was still performing basic binary searches, just in a different way. That was not my intent, but that's what happened. Towards the end, I used some of the indirect information Jon gave me to help with the last few remaining numbers. I was able to get Jon talking without asking yes/no questions, and based on the context of what he was saying, and the numbers remaining, I was able to accurately guess the final number. I had the leading score of six questions and zero false bug reports.

5) Our fifth contestant was Paul Holland (tester). I had the number. Paul, like Julian, used a straight binary search and solved it in eight with zero false bugs, but Paul was in a second session and had no prior knowledge of other contestants. He came to the same conclusion right away. Fast. Clean. Simple. Repeatable. Paul is a constant critic (along with Julian) about those of us who did not stick with the binary search. He kept pointing out, that even if we did better once; over time, we would do worse then someone who used the binary search. He is most likely correct.

6) Our sixth contestant was Amber Mikesell (tester). I had the number. Amber started with the binary search, and then tried to get creative with the smaller solution space. She switched to asking questions like, "Is the number divisible by 6?" and "Is the number divisible by 4?" Towards the end, she had a 50/50. Because of pressure (from me), she made a guess and filed a false bug report. She got it in seven with one false bug report. Her defense was that she didn't want to hold the release and took a risk to ship the product. I liked that.

7) Our seventh contestant was Tim Coulter (current Florida Tech student and probably my future boss given his personality and intelligence). I had the number. Most notable in Tim's technique is that he took risks. He combined techniques of those before him using binary searches and creative questions. Paul considered him very lucky with the results of some of his questions. After question five, Tim had more then a couple of possible solution numbers left. Eventually, he asked if the number was 32. He was correct. When I told him it was a lucky guess, he responded that his logic was based on my ability to randomly select a number. Of the numbers remaining, 32 was the only number not a multiple of 10. His logic was that I would not randomly select a multiple of 10. Looking back at all my selected numbers, he was correct. Tim's technique earned him a score of six questions and zero false bug reports. A tie for first place.

Paul and Julian pointed out (after Tim's attempt) that it would be possible to try to eliminate up to 28 extra numbers in your first guess (or two) and still not increase your maximum binary search result. This would essentially allow the possibility of having less than seven guesses if you were lucky in those lopsided guesses. This maximizes the chance of lowering the number of guesses while not increasing your worst case scenario.

It's a fun little challenge. Try it and see how you do.
[/textile]

Browse by topic:

Comments

My favorite game that my English teacher had going with us for probably half the semester one year was called Molly and Ned. The object of the game was to figure out the rules of the game. This was done by asking yes or no questions. The one disadavantage of the game is that it can only be played once per person. Once the rules are known, the game is over. Of course, it took our class of sixth graders about 3 months to figure them out. The rules are posted below under a spoiler tag.

-------------------SPOILER TAG: HERE ARE THE RULES--------------------

The student asks questions of the teacher and the teacher answers yes or no, based on the first letter of the last word in the question. If the first letter of the last word starts with a letter from A to M, the answer is "yes". If the word starts with a letter from N-Z, the answer is no. Hence the name of the game, "Molly and Ned". So, the question "Is your name Ned?" would be answered "No." The question "Is your name Molly?" would be answered "Yes."

So if Amy asked if the number had curves, and the person with the number said "it depends on the font", she'd know it contains either a 4 or a 7... ;-)

Fascinating game!

I just did this with 3 of my testers and all three of them started with the same question but then went off in totally different directions. The main point that I learned from doing this is that people solve problems differently but if we can see how others solve problems it will in turn make us better problem solvers.

Peggy Collier