Description
One of the primary reponsibilities within every software production
company is that of the Software Tester. There are a large variety of
approaches to testing software that you will learn about in future courses.
In the meantime, it is the goal of this assignment to introduce a strategy
for testing that you can adopt in testing your own applications.
Assignment
You have been tasked with the responsibility of thoroughly testing three
applications that were developed by another programmer within your company.
The programmer created 4 versions of each application and needs to figure out
which is the correct version. There is exactly one correct version of each
application. However, your company needs you to thoroughly document what bugs
you find in the other three versions.
IMPORTANT: Any "bugs" you find that are the same
between ALL the copies of an application are "features", not "bugs" :-D
Applications
Dice Roller (Dice1, Dice2, Dice3, Dice4)
This application rolls two dice that could be used for a variety of computer
games. The values of each of the two die are one (1) through six (6). You
can roll again by selecting 'Y' or 'y' when prompted, any other value will
imply no.
Rock, Paper, Scissors (Rock1, Rock2, Rock3, Rock4)
This application plays Rock, Paper, Scissors against the user. The computer
will randomly generate its own guesses, and the user can type 'R', 'P', or
'S' in each round to play. At the very beginning of the application, the
user can enter the number of rounds they wish to play. When the total number
of rounds are over, the computer will display the number of wins for each
the computer and the user. Determining the winner follows the standard
RPS rules, Rock beats Scissors, Scissors beats Paper, Paper beats Rock.
Three Word Sorter (Sort1, Sort2, Sort3, Sort4)
This application sorts three words at a time that the user supplies. If you
would like to continue to enter in sets of words, you can respond with 'y' or
'Y' when prompted. The comparison used is a dictionary-based comparison:
A < a < B < b < ... < Z < z. Also, shorter words are
less than longer words. In example, the words {b a c} would be sorted
as {a b c}, {a A B} would be sorted as {A a B}, and {aa a aaa} would be
sorted as {a aa aaa}.
Testing Strategy
It is your responsibility to thoroughly test each application to determine
precisely what is wrong with it and which is the correct version. It is
recommended that you following the test development strategy introduced in
the first lab.
Use input and output redirection to create testing files
to run each application. Combine this with extending your makefile to run
testing commands - remember, when creating targets, dependencies and commands
in a makefile - any command can be used, it need not be a command for
compiling some program. Think about using a "tester" target that runs a set
of commands to run the applications through your test suite.
You must automate as much of the testing system as you can. Imagine if there
were 100 applications to test, how might you use a makefile to help simplify
your test runs?
Writeup
Obviously, there is no application development for this homework, we are only
interesting in ensuring that you can create testing suites and can find
errors in existing programs without know much about how they were written.
You should turn in a README document that describes the following:
For each application
- Which version was correct.
- For each of the other three - describe specifically what was wrong. You
will need to be very specific here, if your comments are too general, they
will be marked wrong.
- For each of the other three - what test(s) identified the error.
- What was your basic testing strategy? Did you follow some general pattern?
If so, what pattern did you use? How confident do you feel that your tests
thoroughly exercised each application and found all of the bugs?
- TURN IN:Entire test suite, makefiles, testing files, etc. that
you used to test the application.
Be specific, you must demonstrate a thorough understanding of what was
incorrect in each application, answers such as "doesn't sort correctly" would
not be specific enough. You must provide enough detail that the developer
could easily locate and fix the error.
Files
You can copy the applications to test from the following directory:
<![CDATA[
/afs/umbc.edu/users/d/a/dana3/pub/CMSC202/hw1/
]]>
Use the following command to copy all the files from hw1 to your local directory (the current directory):
<![CDATA[
cp /afs/umbc.edu/users/d/a/dana3/pub/CMSC202/hw1/* .
]]>
The '*' means "all the files in this location" and the '.' means the current
directory. All of the files to test are executable files, NOT code, you cannot
compile them yourself, they have already been compiled, you simply have to
run them (just as you would run a.out).
Your makefile must have its first target as "Hw1" and it must run all of
your test cases. When the grader runs "make Hw1", this target must execute
and all of your tests must be reproducable.
Grading
The grade for this homework will be broken down as follows. A more detailed
breakdown will be provided in the grade form you receive
with your homework grade.
100% - Correctness
This list may not be comprehensive, but everything on this list will be
verified by the graders.
- Identified the correct version
- Correctly identified the errors in the other versions
- Designed and Used a thorough test suite
- Used a makefile to assist in the automation of the testing suite
Homework Submission
What to Submit UPDATED!
You should submit the following files:
- README - a text file describing what bugs you found, how you found
them and your general testing strategy (as described above)
- makefile - a file that has its first target as "Hw1" so that it will
make with the submitmake command, this target should cause all tests that you
ran on the applications to run, you will need to submit all of the application
programs for this to work
- test input - any files of input that you used for your test cases
- executables - submit the 12 executables so that you can make your
project remotely
Steps for Submission
- submit all files
- submitls to verify they are in the remote directory
- submitmake to build your files remotely
Assuming you've used the recommended file names, then
to submit your homework, type the command
<![CDATA[
submit cs202 Hw1 files ...
]]>
The order in which the files are listed doesn't matter. However, you must make
sure that all files necessary to compile your project (using the makefile)
are listed. You need not submit all files at the same time. You may resubmit
your files as often as you like, but only the last submittal will be graded and
will be used to determine if your project is late. For more information,
see the projects page on the course website.
You can check to see what files you have submitted by typing
<![CDATA[
submitls cs202 Hw1
]]>
Be sure to build your project once it has been submitted using the submitmake
command, so that you know that all of the files are there and are the most
up-to-date versions:
<![CDATA[
/afs/umbc.edu/users/d/a/dana3/pub/CMSC202/submitmake cs202 Hw1
]]>
More complete documentation for submit and related commands can be found
here.
Remember -- if you make any change to your program, no matter how
insignificant it may seem, you should recompile and retest your program before
submitting it. Even the smallest typo can cause compiler errors and a reduction
in your grade.
Avoid unpleasant surprises!
Be sure to use the submitmake
utilities provided for you to compile, link and run your program after you've
submitted it.