Deadline: 25 Nov, midnight so that teams have time to review & fix problems.
This week, every team should have deployed their application to a cloud service.
URLs are posted in the class projects sheet: https://bit.ly/isp2023-projects
Report Bugs
- Google sheet Bug Hunting and
- On the project’s Issue Tracker
Assignment
Try to find defects (bugs) in other teams’ apps, using the cloud-deployed version.
What’s a Defect?
- Missing & Incomplete Functionality: any feature listed in the team’s project proposal that is missing or incomplete in the actual app.
- Functionality/correctness: App returns incorrect or invalid data or fails to retain data when it should. “Data” includes what is shown on a web page.
- Security errors.
- Disclosure of confidential settings. Can you get it to print the settings on an error page?
- Data exfiltration, meaning to access data you should not be able to access
- if you are more ambitious, try fuzz testing (sending random requests)
- please try SQL Injection or Javascript Injection
- Authorization errors (a subset of “Security” errors)
- being able to modify data submitted by someone else
- access a restricted page without login (“authorization bypass”)
- you don’t have to use the web UI! Discover what URLs that app uses (including looking at source code) and try submitting GET/POST/PUT/DELETE requests direct to those URLs without authorization. This may work well with REST web service urls.
- try to access the admin account (guess common passwords)
- Appearance: Mistakes in the UI, including spelling and formatting errors.
- Navigation errors:
- Links that don’t work or go to the wrong page.
- Inability to navigate, such as cannot get back to the main page without using browser Back button (remember KU Polls?). This is a “Usability” error.
- Any valid request that returns a 404 Not Found response.
- Apllication Error or Unresponsiveness: anything that causes the app to become unresponsive, return an exception page, or a 5xx status code page.
- Anything that fails to work.
- Installation instructions don’t work. You tried to install and run it locally using installation instructions in project’s Github repository, but they don’t work. This includes missing steps.
- Other perceived problems
Testing and Reporting
-
Test as many other apps as you want to find defects and earn bug bounties.
-
Reporting Defects: See instruction on the “Bug Bounty” spreadsheet.
Earn Bug Bounty Points
- The first person to report each confirmed, unique “bug” or “defect” related to functionality or installation earns 1 point.
- Security defects earn 2 points.
- Usability, Appearance, and Other defects earn 1 point if deemed significant by instructor.
The person who finds the most defects will earn an extra 5 points.
Points count as an extra homework score.
Validation by Development Team
Promptly review each issue. Apply a label to the issue (Github lets you add labels to issues).
Development Team apply these labels:
- Confirmed - if you confirm its a bug that hasn’t been reported before.
- Duplicate - the issue duplicates another issue. Add a comment like “Duplicates #18” to the issue, and close it.
- Need Clarification - request for more information is needed, because the problem isn’t clear or you can’t reproduce it. Update the label (Confirmed, Not a Defect, etc.) when you understand the issue.
- Cannot Reproduce
- Not a Defect - If the team concludes that its not actually a defect. You should explain why its not a bug in your comment closing the issue.
- If the bug reporter still believes its a defect, he may reopen the issue once. He may also contact a TA to arbitrate, if necessary.