The deterioration of unmanaged bug bounties
Being a developer is hard. Taking care of a security related bug is even harder. There are conflicts in agreements, issues in disclosures, lack of technical information and and overall dissatisfaction in the way people report bugs.
Participating in bug bounties and developing web applications have been some of my best hobbies for the last 3-4 years and I have been on both sides: receiving security reports and sending them out. Not only do I wish to speak with credibility, but more so wish to surface an issue which has been plaguing the bug bounty community recently.
Why? What's wrong with bug bounties?
Aren't they a great way to find bugs that endanger your application? Yes, but a bug bounty can also do great harm to the production of software and the productivity of developers.
Problem 1: Lack of risk validation and proper rulesets
If you want integrity in your bounty program, you're going to have to buckle down and make some definite rules that apply to the researchers that submit bugs. These rules should not be consistently bent, but based on your responsible judgement and a bugs severity - feel free to break them if need be.
Bugs including:
- Mixed Content on Page
- Clickjacking (X-Frame-Options) header missing
- Pointless enumeration bugs (where risk to users/web app is minimal)
- Directory Listing
- Server Banner Disclosure
- Internal Server Error logs
- Referrer Leakage
- Logout CSRF / Contact form CSRF
- Sandboxed User-content Domain XSS
- "Best practices" i.e. HTTPS cert weaknesses on domains with no user data
- URL redirection bugs with no relevant exploitation vectors
- Vulnerable jQuery Library (without PoC)
- Bruteforcable Login Pages that use Basic Auth
- Autocomplete enabled forms (e.g. login pages) - see https://hackerone.com/reports/109 (I still cannot believe $200 was awarded for a non fatal autocomplete bug)
should usually not be awarded bounties, unless you as the developer, can identify a possible risk from not fixing what is reported from the list above.
- For example, if a security researcher submits a bug regarding the lack of "X-Frame-Options" without justifying the risks, or without you, as a developer being able to see the risks of not including the header - simply ask the researcher for how the vulnerability could be maliciously exploited by an attacker.
If the security researcher can prove or justify his submission and you are satisfied that the bug is of legitimacy - then please do award the researcher.
Else, kindly tell the researcher that since the finding does not pose any risk identified by yourself as the developer or by the researcher, it will not be awarded a bounty.
Tone, sincerity and gracefulness really do matter when responding to researchers. When a researcher participates in a bounty, they are essentially taking a risk. At the price of their time, they are willing to test your application for security related bugs. For them, participating in a bounty is an investment.
Problem 2: Bounty Ethics
I'll be honest with you. Most bug bounty hunters are there for the money or for the fame. Regardless, they don't care too much about how they get there. All they care about is the end result. Whether that be bounty or fame.
What's wrong with that?
- Excessive use of tools such as Web Application Scanners, cough acunetix.
- Lack of morale, ethics and care for the web application and/or its developers/users.
- Rush - badly worded emails, lack of PoC's, lack of information.
- Invalid reports, plagarised content and overall lack of manners.
In fact, recently, I saw this slide and started cringing on the inside:
"A better approach" Mixed content? Click Jacking? When don't pay don't invest much time!! Remember even a CJ can give you a HOF? Wow, if this is how what upcoming bounty hunters are being taught, that's quite a disgrace.
@m_ksimka @PaulM @prakharprasad They All Do This
— Egor Homakov (@homakov) May 17, 2014It's obvious that other well established security researchers and developers agree about the absurdities in security bug reports.
Problem 3: Extreme Gamification and Competition
A certain level of gamification and healthy competition is all well and fine when it comes to bug bounties - but as of late, especially in Asian countries, bounties have become a form of extreme competition. The more HoF's a security researcher has, the more credit/reputation he/she has in a community. Here's some evidence for my claims:
Until someone snaps....
The urge to get onto a hall of fame for bounty hunters like shown above, is quite high - and hence their ethics and morale to get there is quite low. Others just really need the money. The extreme motive, competition and gamification causes a great inconvenience for those dealing with security reports.
Some reporters are current blackhats who want to make extra money or gain cred amongst their peers. Here, take a look at this:
What now?
Fixing a broken community?
Transparency - Bug bounties, reports and security researchers need to be transparent. If reporter 'x' get's a bug on website 'y', website 'y' should release his/her report to the public, and/or declare the reason why that reporter is listed in the HoF.
Management - A filtering process must be applied for every vulnerability, such a process must be strict and should definitely only be bent in extreme/valid circumstances.
Blacklisting - Bounty hunters who consistently break rules or cause harm to your program should be blacklisted.
Detection of Automation - If automated tools are not allowed, some sort of penalty MUST apply to bug bounty hunters if they use such tools, because if no penalty applies, they will not stop.
Developer Involvement - Don't accept a bug because the researcher claims that it is of severity, confirm the severity and then continue on.
Current Solutions
Managed bug bounty system where bugs can be filtered appropriately, and irrelevant reports are dealt with. Free for charities and non-profit organisations.
An unmanaged bug bounty platform. Seamlessly takes care of transparency, and assists heavily with management/bureaucracy. Free, if no monetary reward is being offered.
Thanks @prakharprasad for adding onto my list of ridiculous so called "bugs" that should not be paid for unless a valid security risk is present.