Roblox must make changes after failing to block adults creeping on kids
Australia's eSafety commissioner found Roblox still fails to prevent adults from contacting minors despite previous safety changes, with gaps in profile privacy, connection requests, and parental alerting Roblox has entered a court-enforceable agreement to make changes within three months, including making children's accounts private by default and preventing adult contact without parental consent This marks the first time Australia has required third-party auditing of a platform's child safety
Analysis
TL;DR
- Australia's eSafety commissioner found Roblox still fails to prevent adults from contacting minors despite previous safety changes, with gaps in profile privacy, connection requests, and parental alerting
- Roblox has entered a court-enforceable agreement to make changes within three months, including making children's accounts private by default and preventing adult contact without parental consent
- This marks the first time Australia has required third-party auditing of a platform's child safety compliance under the Online Safety Act, with fines up to $49.5 million for violations
- Roblox's in-house AI facial age estimation system has processed 338 million records but faces scrutiny over data retention (up to 30 days) and the inherent difficulty of ML-based age classification
- The agreement forces Roblox to submit its age estimation tools and safety safeguards to independent verification, ending its ability to self-audit compliance
Why It Matters
This case represents a landmark enforcement action under Australia's Online Safety Act, establishing a new precedent for regulatory oversight of AI-driven safety systems on platforms with significant minor populations. For AI practitioners and platform builders, it underscores that algorithmic age estimation and child protection mechanisms are now subject to independent third-party auditing and carry substantial financial penalties for non-compliance. The ruling signals a broader industry shift where self-regulation is no longer sufficient—platforms must prove safety effectiveness through transparent, externally verified systems.
Technical Details
- eSafety testing revealed that children's profiles (account names, connection lists, avatar images, interests) were visible to all users with no privacy restriction options, and connection requests from adults did not trigger parental alerts
- Roblox's facial age estimation system is built entirely in-house using a high-throughput machine learning layer that evaluates whether ongoing user behavior remains consistent with their estimated age over time
- The company acknowledged that "a single accuracy number is not very meaningful" for age estimation, citing ranking quality, calibration, stability over time, and behavior near age-band boundaries as additional evaluation dimensions
- The system has processed 338 million age-check records to date, with fused models producing "high-confidence estimates," though selfie data may be retained for up to 30 days before automatic deletion
- Upcoming mandated changes include default-private children's accounts, consent-gated adult-to-minor contact across the platform, improved reporting mechanisms, and post-report outcome notifications to users
Industry Insight
- Platforms with significant minor user bases should proactively implement default-private account settings for users under 16 and build consent-gated communication flows, as regulatory bodies are moving from voluntary compliance to enforceable mandates with independent auditing
- AI-based age estimation systems will face increasing scrutiny not just on accuracy metrics but on calibration, temporal stability, and data retention practices—companies should prepare for third-party audits of their ML pipelines and ensure transparent documentation of model evaluation criteria
- The $49.5 million penalty framework under Australia's Online Safety Act sets a precedent that will likely influence other jurisdictions; companies operating globally should design safety systems with the strictest regulatory standard in mind rather than patching gaps reactively
Disclaimer: The above content is generated by AI and is for reference only.