Belgian APD methodology for website audits not without flaws

Within the most recent decision about the use of cookies by a Belgian publisher, the Belgian APD (BAPD) not only clarified its expectations for cookie compliance, which I covered in this post, but also offered insight into its methodology for putting websites to the test.

BAPD mentions the following tools throughout the decision:

• Website evidence collector (WEC): an open source web scraper, developed by the European Data Protection Supervisor (EDPS), to automatically capture cookies and tracking on a website domain
• Cookie manager: an open-source browser plug-in, developed by Rob Wu, to capture cookies while manually visiting a website.
• OneTrust classification: public directory of cookies by OneTrust.
• Cookiebot classification: proprietary classification by Cookiebot offered as a freemium.

This patchwork of tools tells us two things about the approach of the Inspection Service (“IS”). First, IS primarely makes manual observations as WEC doesn’t handle CMP choices. Second, it has not spent any money on a license for scanning software or cookie databases.

Not surprisingly, the publisher pointed out several flaws about the observations by IS:

• BAPD combines an automated and manual scanning approach without clearly distinguishing both in their observations.
• The manual approach is error-prone (e.g., not clearing cookies and cache prior to scanning).
• The OneTrust and Cookiebot classifications contradict each other and are not based on verifiable sources.
• Observations do not contain a timestamp or specific URLs.

The Litigation Chamber (“LG”) rebuts this criticism by arguing IS can exercise broad discretion over its methodology. IS does not have to justify its choice for specific tooling nor explicitly document technical aspects thereof.

Even if the current methodology can point to actual ePrivacy or GDPR infringements, IS makes it hard to verify the accuracy of observations and replicate findings. A more robust domain scanning approach comprises four building blocks:

• Automatic rather than manual capture of cookies, local storage – often overlooked, but equally important as cookies - and tracking;
• Repeated scanning (e.g., daily) to uncover consistent patterns;
• Testing a range of privacy choices (e.g., rejecting all purposes, accepting analytical purposes only, accepting content recommendations, but not personalized ads);
• Contextualizing the purposes of each request based on a constantly evolving catalogue of domains and service providers.