6 June

Github notified almost 400 thousand people via email

News
Programming

0 min. read

Reading Time: 3 minutes

Today someone used Github’s notification feature to request a merge of their commit. Unfortunately, they tagged 400 thousand people for the ride.

Message from a committer on github that requested pull request and 97 answers in that thread.
Message from a committer on github that requested pull request and 97 answers in that thread.

How notifications work on Github

When someone creates a merge request or pull request they need to have it approved by others to confirm that it’s a valid work and would not break existing systems.

A person who created that code and created this pull request usually tags the main reviewer or in the public repository, a maintainer of the repository to get the work approved faster so that they do not have to make a lot of context switching when working on new tasks.

A pull request is a way for a developer to submit code changes to a project for review. Once the code changes are reviewed and approved, they are then merged into the main codebase.

On Github, in order to do that, you need to write @Persons_Name to notify them. This is usually not very dangerous as the list of individuals could not be log as for sure there are some spam filters to prevent thousands of mentions in a single message.

Github group tagging

Some organizations allow for teams and groups within the teams to exist so that they can categorize them via permissions. Unfortunately, they can also be tagged by @ sign. Using @Team or @Team/Group.

Quote from the github's pull request that links to the group and shows count of members at 398463
Quote from the github’s pull request that links to the group and shows count of members at 398463

To form a @Team you need to have an organization that allows for members or for the public repo you needed to introduce change within the last 90 days or so to be part of the tag.

Unfortunately for EpicGames, their repository requires special access and acceptance of a EULA after which you are added as a developer to the organization.

After accepting a EULA you are added to the Epic Games team and given access to all of their repository of Unreal Engine and a few others. This addition is permanent unless you get banned or leave the group.

Unreal Engine developers team

As you might guess so far, someone tagged the whole EpicGames developer’s team within their organization, and because GitHub by default sends everyone emails notifications as well… Well, they got an email, and another one and another…

Unreals engine developers team
Epic Games, developers team on GitHub
https://github.com/orgs/EpicGames/teams/developers

A lot of the notified users decided to join in to message all of those developers again, as once you are subscribed to a thread (even via a mention) you are receiving all updates from this thread until you unsubscribe.

This wild ride took almost 1 hour before this topic was closed. Around 100 messages were sent, most of them were not very insightful but most complained about “unsubscribe” doesn’t work for them.

Github’s Unsubscribe button

Those who tried to Unsubscribe from a thread were met with an unpleasant surprise, as the notification system works like a queue. First, the message is verified if the given user should receive an update. Then it’s added to the queue. Once the message is in the queue there is no coming back, and because of the sheer volume of those emails, the queue was popping, sloooowly.

We have reached out to GitHub for comments but we didn’t receive a statement yet, probably they got a need to go through all the angry people who did receive an email from this thread.


Author

Tomasz Juszczak

CTO /

Technical Lead

Tomasz Juszczak
WARSAW,

What's your story?

Direct contact