CLASP Password Manager
Test Results
COP4331 - Processes for Object Oriented Software Development - Fall 2014
Modification History:
| Version | Date | Who | Comment |
|---|---|---|---|
| v0.0 | 10/22/2014 | Cindy Harn | Created Template / Empty Document |
| v1.0 | 11/19/2014 | Cindy Harn | Imported and modified Test Plan documentation |
| v1.1 | 11/25/2014 | Cindy Harn | Uploaded and updated the entire document |
Team Name: Group 8
Team Website: http://www.cs.ucf.edu/courses/cop4331/fall2014/cop4331-8/
Team Members:
2. Description of Test Environment
4. Actual Testing vs. Planned Testing
5. Description of Individual Test Cases
Overall Objective for Software Test Activity:
The overall objective for the Software Test Activity of the Password Manager is to ensure all core functionality is present. In addition, the test processes will be used to identify any bugs with the interface or logic of the software. This will allow the identification of the most important issues, and prioritize them to determine what needs to be addressed immediately, as well as potential bugs that will be fixed later in the development process. There may also be issues that arise that the group can identify as not worth fixing, or a minor issue that can potentially be addressed before delivering the software. Automated Unit Testing is expected to be used for functionality that allows it to help with quick iteration of the software design.
Reference Documents
SECTION 2: Description of Test Environment
The Test Environment will consist of all hardware and software available to the team at a minimum. In the context of the project, this will include Windows and various Linux distributions as the OS. In addition, the 32-bit and 64-bit editions of each Operating System will be used as well. For hardware, there is an assortment of modern and legacy laptops. For Windows, the team will test against all of the newer releases, including Windows 8.1, 8, 7, and Vista. For Linux distributions, it is planned to be testing against all popular distributions such as Ubuntu, Fedora, Archlinux, and others. The most recent LTS releases of each applicable distribution will be tested. The Linux kernel is expected to be reasonably up-to-date and still supported. This includes the most recent kernel versions as well as all currently supported LTS releases.
As the application is currently targeted for Java supported platforms, the group is planning to test application functionality against all major Java Platform Standard Edition releases on the platforms described above. The EOL Support Roadmap as provided by Oracle will be used in order to guide the team in this testing: http://www.oracle.com/technetwork/java/eol-135779.html
The database and web service for the application will be developed and tested on CentOS 6.5 Minimal installation within a Xen hypervisor, with the additional packages necessary for development, maintenance, backups, and deployment. The hardware is a Xeon series quad-core CPU with 2GB of RAM at our disposal. Disk space is adjustable. For the database, the latest package of MySQL available to CentOS through the official repositories, or the EPEL will be used for testing and deployment.
There will be three types of testing: Unit Tests, User Tests, and Developer Testing. Unit Tests will be performed every time a major feature is merged into the Master Branch of our revision control. If a test fails, this issue should be addressed right away as these tests are meant to verify basic functionality of the app. The goal of the Unit Tests are to maintain a working prototype that encompasses the basic functionality with little issues.
User Testing will occur during each major milestone, as determined by the developers. Bugs or issues found during these testing periods should be minimal and not impact the basic functionality of the application. If issues are found, they should be conveyed to the developers, where they will enter these issues into the issue tracker for investigation at a proper time.
The Developer's Testing will occur around the same time as Unit Tests, when a new feature is merged into the master branch. This type of testing may even occur during the development phase of the new feature, before any commitments are made to the local repository or the master branch.
In the case of a major, or even fatal, error for the application, it is expected to investigate the issue in full detail to determine the issues and any possible workarounds. These types of issues should not be merged into the master branch. With these issues, as we are developing using revision control with feature branches, they should not directly affect the master branch, which should always be free of fatal or major issues. If assistance is needed in investigating the fatal error, the feature branch should be committed to the remote repository for others to clone and debug.
The software can be considered "good enough to deliver" once all major unit tests are consistently passing on all testing platforms. An amount of ten unit test runs of each major unit test on each platform can be considered good enough. In addition, we expect the user and developer's testing to be free of major issues as well. The software cannot be considered "good enough to deliver" if it does not meet these minimum requirements.
The software package will be deemed "good enough to deliver" if there are no known errors related to core functionality. Cosmetic errors or bugs can be acceptable. If there are performance issues associated with the software, such as response delays in account creation or retrieval, or the login process, further analysis should be done to determine if the delay can be fixed in a reasonable amount of time, or if it will need to be considered acceptable for submission. All unit tests that test the core functionality and feature set of the software should all pass before delivering the final product.
SECTION 4: Actual Testing vs. Planned Testing
The actual testing consisted of a significant amount of reviewing console logging after each feature was implemented. This occurred on both the client-side Java project as well as the server-side Python project. The logging proved essential when testing or tweaking features. Many of our features could be evaluated by verifying the HTTP response status code to be alerted of immediate issues. A bad request such as HTTP 400 response was a common response that helped determine edge-cases and general issues with a feature. Originally, it was planned to make some use of Unit testing. However, this was not carried out for the most part. We found the logging and manual user interaction, as well as making direct database queries after each feature, to be adequate in verifying the functionality of each requirement.
Additional testing was done by distributing the executable JAR to friends and family. This allowed us to receive a large amount of feedback, mostly regarding UI issues, and gave us the ability to tweak features appropriately.
Additionally, the back-end was tested within a browser, as the endpoints were RESTful design and requests to the service could be made easily with a browser extension. This allowed direct and instant feedback on a back-end change without having to wait for the Java application to recompile. This complemented the development style of the back-end, as the Python code could be updated on-the-fly and changes would be seen immediately. This was a test method that was not previously planned, but proved to be one of the most valuable methods of testing functionality.
SECTION 5: Description of Individual Test Cases
| Test Case #1 | |
|---|---|
| Objective | This test will verify functionality of account creation. |
| Description | This test will be verifying the submission of a valid Username and Master Password. |
| Conditions | Test will be run under "Account Creation" mode. |
| Expected Results | The username is expected to be an alphanumeric between 1-100 characters. It will have an algorithm to check whether the username submitted fits the constraints and prompts otherwise. The Master Password will have a separate test to check the requirements being given for it. However, this test will simply ensure proper entry into the database. |
| Who | Cindy Harn |
| Date | 11/25/2014 |
| Environment | Windows, Linux |
| Result | PASS - The program verified that usernames consists of alphanumeric characters from the range of 1-100 inclusive and that a valid Master Password was entered. |
| Test Case #2 | |
|---|---|
| Objective | This test will verify functionality of Recovery Mode. |
| Description | This test will be used to ensure that the secret answer is validated in an acceptable manner and that a password reset function is provided to the user after successfully answering the secret question. |
| Conditions | Test will be run under "Reset" or "Recovery" mode. |
| Expected Results | This test will ensure that after the user has entered "Recovery Mode" that the correct Secret Answer is validated and a reset option for the Master Password is provided. The input for the Secret answer will be case insensitive and space insensitive. The secret answer provided will be one-way hashed and queried for against the database table for the user's Secret Answer. If the two Secret Answer's match, then the user is authenticated. All other inputs will not be authenticated. |
| Who | Mathew Church |
| Date | 11/25/2014 |
| Environment | Windows, Linux |
| Result | FAIL - This requirement was not met due to obvious security risks. |
| Test Case #3 | |
|---|---|
| Objective | This test will verify functionality of Master Password Reset Mode. |
| Description | This test will ensure that the Password Recovery Mode takes an input with a "strong" Master Password and is stored correctly in the database. |
| Conditions | Test will be run under "Reset" or "Recovery" mode. |
| Expected Results | This test will take the input of the new Master Password for the authenticated user, and replace the old one in the database after being one-way hashed. A query to the database with the one-way hashed Master Password input will verify this. |
| Who | Mathew Church |
| Date | 11/25/2014 |
| Environment | Windows, Linux |
| Result | FAIL - Functionality does not exist due to obvious security risks. |
| Test Case #4 | |
|---|---|
| Objective | This test will verify a password submission to the existing list. |
| Description | This test will take 3 field submissions - Username, Password, and Account Location. We will verify that the resulting data in the database is exactly that of the input, case-sensitive. In this case, as the Username and Password can be for a number of services or applications, the fields will not be restricted to a minimum password strength or limited to a subset of characters. |
| Conditions | Test will be run under "Logged In" or "User" mode. |
| Expected Results | A SQL query will be run to verify these fields have a correct entry in the Password List Table. |
| Who | Michelle Edwards |
| Date | 11/25/2014 |
| Environment | Windows, Linux |
| Result | PASS - Fields were not limited to restrictions and entries submitted are case-sensitive. |
| Test Case #5 | |
|---|---|
| Objective | This test will verify that the checking of a sufficiently strong Master Password is entered upon creating an Account. |
| Description | This test will be checking against the Master Password creation field. The goal here is to ensure that only passwords of a sufficient strength can be entered. |
| Conditions | Test will be run under "Logged In" or "User" mode as well as under the "Account Creation" mode. |
| Expected Results | The results should throw an exception upon Account Creation that prompts for re-entry of a stronger password if the Master Password is considered "weak". A "weak" password is any password that is not considered "strong". The requirements for a "strong" password will most likely change over the course of the project. However, this test will begin by ensuring that only Master Passwords with 10 or more alphanumeric characters are accepted as a Master Password. |
| Who | Michelle Edwards |
| Date | 11/25/2014 |
| Environment | Windows, Linux |
| Result | PASS - Successfully prompted the user if the Master Password entered was too short or contained illegal characters. |
| Test Case #6 | |
|---|---|
| Objective | This test will verify a credentials submission to the existing list. |
| Description | This test will take three field submissions - Username, Password, and Account Location. We will verify that the resulting data in the database is exactly that of the input, case-sensitive. |
| Conditions | Test will be run under "Logged In" or "User" mode. |
| Expected Results | The results will be queried from the SQL database. The fields that hold the Username, Password, and Description should match exactly that of the original input. |
| Who | Thomas Bergens |
| Date | 11/25/2014 |
| Environment | Archlinux(Linux), Java JDK 1.7 |
| Result | PASS - GET requests successfully added accounts to the authenticated user. |
| Test Case #7 | |
|---|---|
| Objective | This test will verify a credentials removal to the existing list. |
| Description | This will test the functionality of a removal feature within the software. The removal feature will have the ability to remove one set of credentials from the user's list. |
| Conditions | Test will be run under "Logged In" or "User" mode. |
| Expected Results | Upon submitting a removal of a credential set, a SQL query will be used to query the Username, Password and Description within the Credential List table. There should return no results, unless there are multiple entries of exactly the same credentials. This can be considered acceptable to the test, unless an "extra" feature is added to remove duplicate entries for the user. |
| Who | Thomas Bergens |
| Date | 11/25/2014 |
| Environment | Archlinux (Linux), Java JDK 1.7 |
| Result | PASS - The DELETE request successfully removed a provided account ¡§id¡¨ if the user was authenticated. |
| Test Case #8 | |
|---|---|
| Objective | This test will verify a successful edit for a credentials entry within the credentials list. |
| Description | This will test the functionality of the edit feature within the Credentials List. The edit feature will allow a user to change the Username, Password, or Description fields in the case they need to update their credentials. |
| Conditions | Test will be run under "Logged In" or "User" mode. |
| Expected Results | Upon editing a credential set, a SQL query will be run to query both the old credentials and new credentials within the Credential List table to ensure that the new entry completely replaces the old entry. |
| Who | Mathew Church |
| Date | 11/25/2014 |
| Environment | Windows, Linux |
| Result | PASS - Program updated changes on the server and the changes could be viewed from another system upon login. |
| Test Case #9 | |
|---|---|
| Objective | This test will verify the destruction of a main account and it¡¦s list of credentials. |
| Description | This will test the functionality of a removal feature within the software. The removal feature will have the ability to remove the main account, and all objects associated with it from the server. |
| Conditions | Test will be run under "Logged In" or "User" mode. |
| Expected Results | Upon submitting a removal of a main account, a SQL query will be used to query the the User object and all object with the User as a foreign key. This should return no results. This can be considered acceptable to the test, unless an "extra" feature is added to remove duplicate entries for the user. |
| Who | Thomas Bergens |
| Date | 11/25/2014 |
| Environment | Archlinux (Linux), Java JDK 1.7 |
| Result | PASS - The DELETE request successfully destroyed the authenticated account, along with all of its associated account objects. |
| Test Case #10 | |
|---|---|
| Objective | This test will verify the password update feature of a credentials. |
| Description | This will test the update password function of a particular credential. Upon clicking the save function of an account, we expect that account to be updated with the new password. |
| Conditions | Test will be run under "Logged In" or "User" mode. |
| Expected Results | Upon submitting an update of a credential password, a SQL query will be used to perform an UPDATE query for the credential's id within the Credential List table. The updated credentials should be returned in the HTTP response with the new password. This can be considered acceptable to the test, unless an "extra" feature is added to remove duplicate entries for the user. |
| Who | Thomas Bergens |
| Date | 11/25/2014 |
| Environment | Archlinux (Linux), Java JDK 1.7 |
| Result | PASS - The UPDATE HTTP request for a specified account list "id" successfully performed an UPDATE query for that id within the database if the owner of the account is authenticated. |
| Test Case #11 | |
|---|---|
| Objective | This test will verify the "login" function of the software. |
| Description | The test will ensure that an existing user is authenticated to the service and can perform future requests. |
| Conditions | Test will be ran upon the start of the program. |
| Expected Results | When the user enters the username and master password, a password hash will be sent, along with the username, as a POST HTTP request to the server to generate an authentication token. The authentication token will be returned in the response body in JSON format for future API calls. |
| Who | Thomas Bergens |
| Date | 11/25/2014 |
| Environment | Archlinux (Linux), Java JDK 1.7 |
| Result | PASS - The POST request with the provided username and hashed master password successfully returned an authentication token. This authentication token can be used for all future requests by providing it in the HTTP request header without issue. |
PASS - All of the functionality that was included in the final product passed their respective test cases. The only cases to fail were those cases in which the functionality was removed due to security risks that the group was not comfortable making for the functionality. In fact, the group has not yet stumbled across a case where the program fails except when trying to use functionality not included in the final product.
This page last modified on November 25, 2014.
Please do not reproduce this page.