The following contributors got their Debian Developer accounts in the last two months:

• Teus Benschop (teusbenschop)
• Nick Morrott (nickm)
• Ondřej Kobližek (kobla)
• Clément Hermann (nodens)
• Gordon Ball (chronitis)

The following contributors were added as Debian Maintainers in the last two months:

• Nikos Tsipinakis
• Joan Lledó
• Baptiste Beauplat
• Jianfeng Li

Congratulations!

Today, the Debian Project pledges to donate $5,000 to the GNOME Foundation in support of their ongoing patent defense. On October 23, we wrote to express our support for GNOME in an issue that affects the entire free software community. Today we make that support tangible.

"This is bigger than GNOME," said Debian Project Leader Sam Hartman. "By banding together and demonstrating that the entire free software community is behind GNOME, we can send a strong message to non-practicing entities (patent trolls). When you target anyone in the free software community, you target all of us. We will fight, and we will fight to invalidate your patent. For us, this is more than money. This is about our freedom to build and distribute our software."

"We're incredibly grateful to Debian for this kind donation, and also for their support," said Neil McGovern, Executive Director of the GNOME Foundation. "It's been heartening to see that when free software is attacked in this way we all come together on a united front."

If GNOME needs more money later in in this defense, Debian will be there to support the GNOME Foundation. We encourage individuals and organizations to join us and stand strong against patent trolls.

In 2012, the Debian Project published our Position on Software Patents, stating the threat that patents pose to Free Software.

The GNOME Foundation has announced recently that they are fighting a lawsuit alleging that Shotwell, a free and Open Source personal photo manager, infringes a patent.

The Debian Project firmly stands with the GNOME Foundation in their efforts to show the world that we in the Free Software communities will vigorously defend ourselves against any abuses of the patent system.

Please read this blog post about GNOME's defense against this patent troll and consider making a donation to the GNOME Patent Troll Defense Fund.

The following contributors got their Debian Developer accounts in the last two months:

• Keng-Yu Lin (kengyu)
• Judit Foglszinger (urbec)

The following contributors were added as Debian Maintainers in the last two months:

• Hans van Kranenburg
• Scarlett Moore

Congratulations!

The Salsa admin team provides the following report about the failed migration of the Docker container registry. The Docker container registry stores Docker images, which are for example used in the Salsa CI toolset. This migration would have moved all data off to Google Cloud Storage (GCS) and would have lowered the used file system space on Debian systems significantly.

The Docker container registry is part of the Docker distribution toolset. This system supports multiple backends for file storage: local, Amazon Simple Storage Service (Amazon S3) and Google Cloud Storage (GCS). As Salsa already uses GCS for data storage, the Salsa admin team decided to move all the Docker registry data off to GCS too.

Migration and rollback

On 2019-08-06 the migration process was started. The migration itself went fine, although it took a bit longer than anticipated. However, as not all parts of the migration had been properly tested, a test of the garbage collection triggered a bug in the software.

On 2019-08-10 the Salsa admins started to see problems with garbage collection. The job running it timed out after one hour. Within this timeframe it not even managed to collect information about all used layers to see what it can cleanup. A source code analysis showed that this design flaw can't be fixed.

On 2019-08-13 the change was rolled back to storing data on the file system.

Docker registry data storage

The Docker registry stores all of the data sans indexing or reverse references in a file system-like structure comprised of 4 separate types of information: Manifests of images and contents, tags for the manifests, deduplicaed layers (or blobs) which store the actual data, and lastly links which show which deduplicated blogs belong to their respective images, all of this does not allow for easy searching within the data.

The file system structure is built as append-only which allows for adding blobs and manifests, addition, modification, or deletion of tags. However cleanup of items other than tags is not achievable within the maintenance tools.

There is a garbage collection process which can be used to clean up unreferenced blobs, however according to the documentation the process can only be used while the registry is set to read-only and unfortunately it cannot be used to clean up unused links.

Docker registry garbage collection on external storage

For the garbage collection the registry tool needs to read a lot of information as there is no indexing of the data. The tool connects to the storage medium and proceeds to download … everything, every single manifest and information about the referenced blobs, which now takes up over 1 second to process a single manifest. This process will take up a significant amount of time, which in the current configuration of external storage would make the clean up nearly impossible.

Leasons learned

The Docker registry is a data storage tool that can only properly be used in append-only mode. If you never cleanup, it works well.

As soon as you want to actually remove data, it goes bad. For Salsa clean up of old data is actually a necessity, as the registry currently grows about 20GB per day.

Next steps

Sadly there is not much that can be done using the existing Docker container registry. Maybe GitLab or someone else would like to contribute a new implementation of a Docker registry, either integrated into GitLab itself or stand-alone?