A THINK TANK FOR MALICIOUS PRACTICES ONLINE

Transcription

1 A THINK TANK FOR MALICIOUS PRACTICES ONLINE 10 POINT COMMENTARY ON TAG S PROPOSED THREAT LIST APPROACH TO REDUCE AD FRAUD a botlab.io publication Mikko Kotila, Principal / Chairman Amit Phansalkar, Principal Data Scientist Sachin Gaur, Security Advisor ALL RIGHTS RESERVED 1 of 10

2 ABOUT Botlab.io is a Foundation focused on researching and publishing malicious practices in the digital media supply chain. A non-profit organisation, botlab.io provides independent insight, opinion and resources for the benefit of the entire digital media eco-system. botlab.io s agenda is founded on the principles of cultivating transparency and mutual trust between advertisers, suppliers and internet users. botlab.io is constituted of a volunteer team who are experts and thought leaders in fields including media technology, data science, information security, networking and mathematics. botlab.io activities include research and analysis that would be hard or impossible for existing industry parties to focus on, including security, threat and vulnerability analysis and testing, forensics and advanced botnet research. botlab.io works closely together with trade bodies, suppliers, advertisers, academia, agencies, and people acting as a source of independent research, insights and knowhow about malicious practices in the digital media supply chain. LEARN MORE ABOUT MALICIOUS PRACTICES IN DIGITAL MEDIA - ALL RIGHTS RESERVED 2 of 10

3 TABLE OF CONTENTS ABOUT 2 TABLE OF CONTENTS 3 OVERVIEW 4 1. BLACKLISTS ARE BETTER FOR SMALLER UNIVERSES 5 2. WE HAVE TO BE ABLE TO DIFFERERNTIATE MORE 5 3. CREATING NEW SITES/DOMAINS IS TOO EASY 6 4. SPAM HAS NEVER BEEN SOLVED WITH BLACKLISTS 6 5. BAD ACTORS ARE INCENTIVISED TO GENERATE NOISE 6 6. BLACKLIST HAS TO BE IP-BASED AND AUTOMATED 7 7. THE LIST HAS TO BE FREELY AND EASILY AVAILABLE 8 8. WE DON T HAVE TO AND CAN NOT RUSH 8 9. WE HAVE TO ENCOURAGE TRANSPARENCY DIGITAL MEDIA MUST REMAIN NEUTRAL 9 : ABOUT OUR MISSION 9 ALL RIGHTS RESERVED 3 of 10

4 OVERVIEW The number of people involved in the process, or the relative importance of those people within the industry are not good indicators of the value an output of a given process may have. The true merit of a given approach to solving a problem, for example rampant ad fraud, is initially in its debatability and ultimately in the test of time. The risk in addressing ad fraud is in moving too fast towards a certain course of action. We know form the past that standards, even bad ones, are hard to die. Once established and adopted, standards tend to impact the way an industry develops long in to the future. The nature of the ad fraud problem is such that while it will be quite easy to get good results in one place, it s very hard to get to an industry-wide result of any kind. Ad fraud is the kind of problem that is easy to move around but hard to make go away. Basically you can make a burglar change their career just because you install a security system in your house. The following pages provide commentary and anecdotal evidence to suggest that the TAG s proposed model to an industry-wide threat list is rushed, based on a limited understanding of the subject matter and has the propensity to make things worse as opposed to improving things. We have not provided the commentary as a criticism to TAG and are supporters of everyone and everything sharing our commitment to make the internet better and more efficient. The points are solely for the purpose of criticising the approach proposed by TAG. Across this document TAG s Threat List program is referenced to as the proposed approach. Read about TAG s program here: ALL RIGHTS RESERVED 4 of 10

5 1. BLACKLISTS ARE BETTER FOR SMALLER UNIVERSES If we assume that there are hundreds of millions of websites and many of them will be bad, then the system we have in place has to facilitate for banning all these bad sites and respond to changes within those sites fast. With 400,000 new malware identified every day, the attack space, and domains related to those attacks, change very fast. Domain names too move fast in the well-established domain aftermarket. Further, the proposed approach fails to acknowledge that the majority of sites are completely bad and will not be good anytime soon. There is no need for voting on sites that no advertiser would want to have their ads on, which can be shown to be the great majority of all sites. These sites are online in the first place for the purpose of separating the advertiser from its money. This kind of inventory is a real problem in ad exchanges and a problem we can do something substantial about. 2. WE HAVE TO BE ABLE TO DIFFERERNTIATE MORE The proposed approach is unlikely to result in something more functional than what could be derived from the 30 or so established blacklist vendors outside of the digital media industry. On the contrary, the proposed approach would likely leave us with an inferior solution compared to one that would simply be based on existing lists. Coupled with ingenuity in putting it all together and maintaining it. The advertising technology industry would like to think it is a spearhead technology innovation industry. Why then limit ourselves to tackle such an interesting problem-set (botnets / ad fraud / malware) with an approach that appears old-fashioned and inefficient. Ad fraud is intimately connected with botnets, and botnets are intimately connected with Malware. During the month it takes TAG to update its list, the adversaries have pushed out a whopping 12 million new kinds of malware from their war chests. ALL RIGHTS RESERVED 5 of 10

6 3. CREATING NEW SITES/DOMAINS IS TOO EASY Our propriety research suggest that adversaries are able to create a single website in minutes, and auto-generate keyword specific versions of that site on a very large scale in matter of hours. These sites can be tailored to fit the IAB tier-1 and tier-2 categories for maximum relevance. Domains are not only the wrong part of the problem, but as an approach can be rendered ineffective too easily by the adversaries, simply by generating more noise. Further, in the proposed approach, it is too easy to get someone else banned. Similarly it s easy to get someone banned again once they ve gone through the painful process of proving their legitimacy. 4. SPAM HAS NEVER BEEN SOLVED WITH BLACKLISTS spam was not solved with black listing. It was solved with advanced science, ingenuity and true global industry-wide collaboration, with much of the work done by independent organisations and in global collaboration. Thanks to great ideas and hard work of the anti-spam community, while spamming is very much alive, we don t see it much in our everyday lives. It s very important to keep in mind that ad fraud is mostly a subset of a larger malware problem. The malware problem is greater today than it has ever been before. If it could be solved or sustainably reduced by using domain blacklists, we would not have such a big (and growing) problem. 5. BAD ACTORS ARE INCENTIVISED TO GENERATE NOISE The proposed approach will encourage adversaries to create tremendous amounts of noise. This is what financial market algo bandits do and this is what Facebook ad fraud companies do. They create tremendous amounts of noise so it would be impossible to find out what are the true intentions of ALL RIGHTS RESERVED 6 of 10

7 the adversaries. Noise makes it hard, if not impossible to measure anything accurately and we are already seeing symptoms of this in the market. For example, due to the obstructive fog the noise creates, it has been easier for many suppliers and sometimes even advertisers to consider ad fraud a cost of doing business. The technique of hiding your true intentions behind an unimaginable volume of noise has been widely used in electronic markets. In the financial markets, there are sometimes hundreds of thousands of fake signals to every actual signal. In effect, noise is the worst enemy of transparency and makes any real transparency impossible to attain. spam, SEO spam and malware have all seen a dramatic increase in amount of noise over the years. 6. BLACKLIST HAS TO BE IP-BASED AND AUTOMATED A list system has to be an objective and 100% automated system system with no human intervention needed or possible. The system must scan everything indiscriminately, and the primary defence blacklist should be IP based. IPs comes before domains in the ad fraud cycle, and will therefore yield superior coverage over the entire ad fraud problem. Additionally the entire IP address universe is known whereas the domain universe keeps continuously changing. Domain aftermarket, renting parked domains, sub-domains and thousands of new top level domain give adversaries virtually infinite possibilities for creating and laundering ever more domains. Domain based blacklisting provides a good secondary line of defence to compliment the primary, IP based defence. ALL RIGHTS RESERVED 7 of 10

8 7. THE LIST HAS TO BE FREELY AND EASILY AVAILABLE A list has to be made freely available to anyone who wants to get access to it. It cannot be limited to members or partners or any other constituency. The data must be accessible in real-time. Vendors can access the data through an API where they can either retrieve updates to an existing dump on their system, or retrieve the entire database. 8. WE DON T HAVE TO AND CAN NOT RUSH Time sensitivity should not be a driving factor in addressing ad fraud and our role in the larger botnet problem. The issues raised here, together with other points, should be investigated, analysed and elaborated in great detail. Time sensitivity makes it hard or impossible to create robust plans and is more relevant when dealing with acute symptoms. Currently we are investigating a chronic condition. We ve had high levels of ad fraud for at least 10 years and virtually no one had a problem with it. Even if it takes us a few years to get it down to a half of what it is now, everyone and everything will be fine. 9. WE HAVE TO ENCOURAGE TRANSPARENCY Lists and any other information and resources about the digital media supply chain, malicious practices in particular, should be publicly available information. A blackbox is the polar opposite of something transparent. Lack of transparency is at the heart of the ad fraud problem and other problems related to malicious practices in the digital media supply chain. In addition, any proposal should focus first on explaining the caveats of the approach, highlighting the known caveats of the system. By providing such ALL RIGHTS RESERVED 8 of 10

9 transparency, we encourage debate and critical peer assessment of the proposed approach. 10. DIGITAL MEDIA MUST REMAIN NEUTRAL Any industry-wide program should have vendor and publisher neutrality as key points of discussion. Further emphasis has to be put on other aspects of neutrality, such as being geographically representative. The future of the media industry is increasingly outside of US, and it is more important than ever to avoid landmark programs that represent one geography more than another. Blacklisting has industry-wide (global) ramifications, and therefore should be deliberated on from a transnational standpoint. RESEARCH BY: Mikko Kotila, Principal / Chairman Amit Phansalkar, Principal Data Scientist Sachin Gaur, Security Advisor Thanks to the botlab.io Advisory Council and all our crypto friends for technical inspiration and for keeping us fighting the right fight. : ABOUT OUR MISSION Depending on the source, the digital media industry is exposed to ad fraud somewhere between 10% and 90% of total investment. Because of the industry-wide nature of the problem, it seems safe to assume that every company in the eco-system is exposed to a certain extent. Yet ad fraud as a subject matter is poorly understood, and requires deep understanding about media technology, malware, security, and the adversary landscape. ALL RIGHTS RESERVED 9 of 10

10 Due to the complexity of the subject matter, the only way (outside of luck) to avoid poor decisions is to develop fundamental understanding about the topic. This requires answering key questions such as: - who are the adversaries? - what kinds of motivations they have? - what kinds of codes they create? - how do those codes get in to the system? - where do those codes originate from? - what do the codes do once in the system? - how do the codes generate cash or other impact? - how are the codes managed and updated? - who are the beneficiaries and how does the money flow to them? When these, and other questions are better understood, creating industrywide ad fraud reduction and prevention solutions and programs becomes far more feasible. Sustainable technology solutions have to be based on indepth understanding of the subject matter. We are working on this research. We want to help the digital media industry develop profound understanding of the relationship it has with botnets and the increasingly digital 21stcentury human culture. We greatly value the opportunity of working together with people who share the same goals with us. LEARN MORE ABOUT MALICIOUS PRACTICES IN DIGITAL MEDIA - A THINK TANK FOR MALICIOUS PRACTICES ONLINE ALL RIGHTS RESERVED 10 of 10