Skip to content

Mailing lists excessive bounce issues ? lei is the solution

Jump to bottom
Neil Armstrong edited this page Sep 2, 2026 · 2 revisions

Mailing lists excessive bounce issues ? lei is the solution

Major email providers such as Gmail and Hotmail are implementing new authentication mechanisms to combat the overwhelming volume of spam. This presents an increasing challenge for mailing list programs to reliably deliver emails.

Google Mail clients have recently experienced a significant increase in the delivery queue from the LKML. This is primarily due to an exponential rise in bounce rates over the past few months.

K. Ryabitsev 🍁

I once again ask you to please stop using gmail for mailing lists.

Attached is a screenshot of *just one* of the 4 nodes delivering various kernel mailing list traffic with a deferred queue of 100,000+ messages, all for gmail.

We are currently sitting on about 500,000 deferred messages, all for gmail. I am very, very close to just banning gmail addresses from large lists.

https://social.kernel.org/notice/AwjNvypdsflO4crYBM

An email bounce occurs when an email message is returned to the sender because it cannot be delivered to the recipient's inbox. This is similar to a letter being returned to you if the address is incorrect or the mailbox is full. Email bounces can be categorized into two main types: hard bounces and soft bounces.

public-inbox

Public-inbox.org offers a robust solution for archiving public mailing list emails. An example of this project in action is lore.kernel.org, which serves as a backup and web API for mailing lists related to Linux and its associated projects.

Public-inbox offers a straightforward webpage for email browsing, a stable web API for referencing patches via Message-Id, and an NNTP endpoint for email collection using Usenet-compatible clients such as Thunderbird.

A comprehensive blog post about how to subscribe via NNTP can be found here:
https://brennan.io/2021/05/05/kernel-mailing-lists-thunderbird-nntp/

lei

The lei utility, whose name playfully alludes to "Lorelei" while standing for "local email interface," serves as a crucial tool for synchronizing public-inbox searches with various email storage solutions. This versatile utility can interface with local mail directories (maildir) or even IMAP servers.

For users preferring a local email setup, integrating lei with a maildir is remarkably straightforward and proves highly effective, particularly when paired with email clients like Mutt. This combination allows for efficient offline access to email and a streamlined workflow within a familiar terminal-based environment.

Beyond local storage, lei offers the valuable capability to upload emails to an IMAP server. This functionality is especially beneficial for users of webmail services like Google Mail. By leveraging IMAP, lei enables users to maintain their preferred email client while still benefiting from the centralized storage and accessibility offered by cloud-based email providers. This flexibility ensures a consistent user experience regardless of where their email is ultimately stored.

Let’s setup lei with IMAP

The lei utility is part of public-inbox, thus it will use the same search syntax as a public-inbox instance like lore.kernel.org.

The first step is to install public-inbox from a package (available on most modern distributions) or by source from https://public-inbox.org/public-inbox.git/

The simplest is to go from sources:

  • git clone https://public-inbox.org/public-inbox.git/
  • cd public-inbox
  • sudo perl ./install/deps.perl lei
  • perl Makefile.PL
  • make symlink-install prefix=$HOME

This makes the lei utility available from your $HOME/bin directory, so make sure to add your $HOME/bin in your PATH.

The first step is to convert your mailing-list subscriptions into public-inbox searches:

You can extend your search with more parameters, please look at https://lore.kernel.org/all/_/text/help/

So now you have your search strings:

  • l:linux-arm-msm.vger.kernel.org AND rt:1.week.ago..
  • l:linux-phy.lists.infradead.org AND rt:1.week.ago..
  • l:u-boot.lists.u-boot-project.org AND rt:1.week.ago..

Now you need to identify where you want the emails to be stored on the imap server, for example you can have a “Lists” directory with each mailing list subdirectory, which gives the following URIs:

  • imaps://imap.gmail.com/Lists/linux-arm-msm
  • imaps://imap.gmail.com/Lists/linux-phy
  • imaps://imap.gmail.com/Lists/u-boot

With the search strings and URIs, you can construct the lei commands from the default common parameters:
lei q --include https://lore.kernel.org/all/ --threads --dedupe=mid --jobs=,2 --augment

Let’s identify the parameters:

  • q: is the query command
  • --include: Is the source public-inbox to “include” in your search, you can provide multiple ones
  • --threads: group messages threads in a same match
  • --dedupe=mid: use message-id ad deduplication strategy
  • --jobs=,2: set any query jobs but only 2 writer jobs to lower the IMAP bandwidth
  • --augment: add mails to IMAP folder, do not remove any existing message

Then for each mailing-list append the URI and the search string, like:

  • -o imaps://imap.gmail.com/Lists/linux-arm-msm 'l:linux-arm-msm.vger.kernel.org AND rt:1.week.ago..'
  • -o imaps://imap.gmail.com/Lists/linux-phy 'l:linux-phy.lists.infradead.org AND rt:1.week.ago..'
  • -o imaps://imap.gmail.com/Lists/u-boot 'l:u-boot.lists.u-boot-project.org AND rt:1.week.ago..'

Which gives the following commands to run:
lei q --include https://lore.kernel.org/all/ --threads --dedupe=mid --jobs=,2 --augment -o imaps://imap.gmail.com/Lists/linux-arm-msm 'l:linux-arm-msm.vger.kernel.org AND rt:1.week.ago..'
lei q --include https://lore.kernel.org/all/ --threads --dedupe=mid --jobs=,2 --augment -o imaps://imap.gmail.com/Lists/linux-phy 'l:linux-phy.lists.infradead.org AND rt:1.week.ago..'
lei q --include https://lore.kernel.org/all/ --threads --dedupe=mid --jobs=,2 --augment -o imaps://imap.gmail.com/Lists/u-boot 'l:u-boot.lists.u-boot-project.org AND rt:1.week.ago..'

When running each command, lei will ask you for a login and password for the IMAP authentication, for Google Mail you should create an application password. Please follow the help at https://support.google.com/mail/answer/185833

The first runs can be long if you already have emails from those mailing-lists on your IMAP server, but then you can just refresh by calling:

  • lei up imaps://imap.gmail.com/Lists/linux-arm-msm
  • lei up imaps://imap.gmail.com/Lists/linux-phy
  • lei up imaps://imap.gmail.com/Lists/u-boot

Or simpler:
lei up –all

Automate lei sync

The simplest way to automatically sync the emails is to create an user systemd service and timer.
Create the files:

  • ~/.config/systemd/user/lei-up-all.service

[Unit]
Description=lei up --all service
ConditionPathExists=%h/.local/share/lei

[Service]
Type=oneshot
ExecStart=/home/narmstrong/bin/lei up --all -q

[Install]
WantedBy=mail.target

  • ~/.config/systemd/user/lei-up-all.timer

[Unit]
Description=lei up --all timer
ConditionPathExists=%h/.local/share/lei

[Timer]
OnUnitInactiveSec=10m

[Install]
WantedBy=default.target

And enable them:
systemctl --user enable --now lei-up-all.timer

You can monitor the service and timer with:
systemctl --user status lei-up-all.service
systemctl --user status lei-up-all.timer

More information about lei can be found on Konstantin Ryabitsev blog posts:

What about the mailing-lists ?

You can now either unsubscribe for unmoderated lists or managed by vger.kernel.org or configure the list to disable delivery for moderated lists, like:

Clone this wiki locally