The Group Chat Nobody Approved — JavaScript Bug Hunt

Briefing

Modelled on Facebook Messenger Kids (July 2019): Facebook notified parents that a design flaw had let children join group chats with users their parents had not approved. A child could start a group with their approved contacts, and those contacts' own approved friends ended up in conversation with children whose parents had never approved them.

This project is a reconstruction. groups.js checks each new member against the chat's creator only — not against every child already in the chat.

Fix createGroup and addMember so every member is approved for every other member.

Bug report

BUG-MK-0719 · Priority: Critical (child safety) · Reported by: trust & safety

approvals.isApproved(child, other) is true when other is on child's parent-approved list. Approval is one-way.

A user may join a group only if, for EVERY current member m, isApproved(m, user) AND isApproved(user, m).

createGroup(approvals, creator, invitees) starts with [creator], considers invitees in order and adds each one that passes the rule against the members added so far; the others are listed in rejected, in order. It returns { creator, members, rejected }.

addMember(approvals, group, user): true without change if user is already a member; otherwise add and return true if the rule passes, else return false and change nothing.

Observed: ben's group put dev in a chat with ana, whose parents never approved dev.

Logs

[mk-groups] create creator=ben members=ana,dev (checked against ben)
[trust] ana <-> dev not approved by either parent; conversation active

The code as shipped

src/kids/groups.js (editable)

// Group chats for the kids app.
exports.createGroup = function (approvals, creator, invitees) {
  var members = [creator];
  var rejected = [];
  for (var i = 0; i < invitees.length; i++) {
    if (approvals.isApproved(creator, invitees[i])) members.push(invitees[i]);
    else rejected.push(invitees[i]);
  }
  return { creator: creator, members: members, rejected: rejected };
};

exports.addMember = function (approvals, group, user) {
  if (group.members.indexOf(user) !== -1) return true;
  if (!approvals.isApproved(group.creator, user)) return false;
  group.members.push(user);
  return true;
};

Read-only context: src/kids/approvals.js.