23 Jun
2022
23 Jun
'22
9:16 p.m.
Hi,
== TL;DR: ==
The BitFolk mailing lists changed address, but important stuff got
copied over. You don't need to do anything.
The new list archives are at:
https://mailman.bitfolk.com/mailman/hyperkitty/
and if you DO want to change any of your settings the main interface
is at:
https://mailman.bitfolk.com/mailman/postorius/lists/
But you'll have to do a Mailman password reset before you can log in
to it and change settings.
You can stop reading now unless you are particularly interested in
the details.
== What ==
BitFolk's announce@ and users@ mailing lists which used to be at
lists.bitfolk.com have now been migrated to
<listname>(a)mailman.bitfolk.comlk.com, which is Mailman 3.
== Why ==
The VM which runs lists.bitfolk.com needs to be upgraded. It runs
Mailman 2, which is a strictly Python 2 application. Python 2 was
EOL in 2020 sometime and is not available in newer releases of
Debian, and so neither is Mailman 2. An upgrade will leave Mailman 2
non-functional.
Running Mailman 3, which is a complete rewrite, is a major
undertaking that I have little experience of and I'm not entirely
happy about it. However, unless we either want to outsource the
mailing lists to another provider or switch to something like
Discourse¹, it seems to be still the best option.
While it may have been possible to install Python 2 and Mailman 2
from source on an otherwise upgraded host, that is a lot of work for
what seems like a dead end.
== Your subscriptions ==
All subscribed addresses have been copied over, which is why you are
seeing this email. You don't need to do anything, but if you do want
to adjust your settings you'll need to do a password reset in
Mailman as your passwords have not been copied over.
The web interface for the lists is at:
https://mailman.bitfolk.com/mailman/postorius/lists/
In the footer of messages to the "users" list it will have a link
and tell you which address you are subscribed with. The "announce"
list does not have a footer so if you're unsure what your subscribed
address is there you'll need to look at the email headers.
== Functional changes ==
=== Headers ===
I'm tired of list postings being rejected or going to spam folders.
We can't keep sending emails on your behalf, i.e. with a From:
address that is yours, because that breaks DKIM. So over on the
"users" list all posts will appear to come from:
From: Your Name via BitFolk Users <users(a)mailman.bitfolk.com>
Cc: Your Name <you(a)example.com>
The Cc header is so that people can still contact you off-list if
they want.
Obviously the List-Id: headers have changed so if you're filtering
email based upon those you will need to update your filters. Sorry
about that.
=== Reply-To: added, no more "users-replyto" ===
We used to have a separate mailing list "users-replyto" which got
all the same emails as the "users" list, but added:
Reply-To: users(a)lists.bitfolk.com
to them for people who preferred to explicitly have replies directed
back to the list.
The main "users" list is going to have an explicit Reply-To: added
now, making "users-replyto" redundant. Anyone who was subscribed to
"users-replyto" has been added to "users".
The reason for this is that I got feedback that despite it being a
long held belief that forcing Reply-To: is bad², these days most
people do prefer it. I'm happy to revisit that decision if people
don't like it, but the "users-replyto" hack won't be coming back.
=== List addresses ===
Obviously the list addresses have both changed from
@lists.bitfolk.com to @mailman.bitfolk.com. This was necessary so
that the older Mailman could continue working while the new one was
set up.
No doubt through muscle memory and address books the occasional
email will still end up being sent to whatever(a)lists.bitfolk.com.
These emails are being redirected to the correct place and I
anticipate keeping that redirection in place forever.
=== List archives ===
The public archives of the "announce" and "users" lists have been
imported into Mailman3's archiver, HyperKitty, at:
https://mailman.bitfolk.com/mailman/hyperkitty/
I tried to port over the existing Lurker archives but I just
couldn't get it to work. Lurker is mostly undocumented and hasn't
seen any development since 2007. This unfortunately means that all
the old archive links that are out there in previous emails and in
the wiki, etc. will break if and when I turn off Lurker on the old
host.
I hope to be able to transplant a static copy of
https://lists.bitfolk.com/lurker/ over to
https://mailman.bitfolk.com/lurker/ so that old links will still
work, with automatic redirection. There doesn't appear to be any way
to map an individual Lurker message link to a HyperKitty link, sadly.
The new HyperKitty archives are fully searchable, so if you do
happen to see a Lurker link in a wiki article please do feel free to
find the same message in HyperKitty and update the article. If you
find it in some other place that you can't edit please do just let
me know and I'll update it. The same goes for any other references
to lists.bitfolk.com.
== Let us know about any issues or suggestions for the lists ==
As I say I'm not very experienced in running Mailman 3 so it's
possibly it might break or be misconfigured in some way that I
haven't yet found. So if you spot anything that you think looks
broken, please do let us know.
Also if you have suggestions for how it might work differently or
look differently, I am interested to hear and try to implement,
though I cannot promise that it is possible.
Certainly I am not in love with the default appearance of the
HyperKitty archives or the colour scheme of Mailman 3's web
interface in general, and it may be possible to improve those.
Thanks,
Andy
¹ Discourse: https://www.discourse.org/
² https://marc.merlins.org/netrants/reply-to-harmful.html and its
counter-argument
https://marc.merlins.org/netrants/reply-to-useful.html
Both of these are over 20 years old so I'm not sure how relevant
they are (or that mailing lists are) for the modern Internet.
It should also be noted that one of the objections to Reply-To:
munging is that it can remove all information about where the
author wants replies to go. In Mailman3's case, if you supply
Reply-To: headers of your own then it adds them, so that objection
doesn't hold.
--
https://bitfolk.com/ -- No-nonsense VPS hosting
_______________________________________________
BitFolk Announcements mailing list -- announce(a)mailman.bitfolk.com
To unsubscribe send an email to announce-leave(a)mailman.bitfolk.com
23 Jun
23 Jun
9:28 p.m.
Didn't take long for the first issue then! 😀
On Thu, Jun 23, 2022 at 09:16:00PM +0000, Andy Smith wrote:
_______________________________________________
BitFolk Announcements mailing list -- announce(a)mailman.bitfolk.com
To unsubscribe send an email to announce-leave(a)mailman.bitfolk.com
So, I said the "announce" list wouldn't have a footer, but it did.
My mistake. I turned it off on the "test" version of that list, but
forgot to do so on the real one.
Should it be removed or do people find it useful to know the details
of the mailing list that even announcements came from?
I want to keep a fairly detailed footer for the "users" list
because I find people do need that to know how to unsub and change
their options, etc. Yes, I know that experienced people know to look
at the headers for that. Repeating that over and over is less
appealing than just sticking it on the bottom of the emails.
It's not going to be an issue for DKIM as on the "announce" list the
only domains that posts is bitfolk.com and the list server can DKIM
sign on behalf of that. On the "users" list the From: header is
changed intentionally so that the list server can DKIM sign.
Thanks,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
24 Jun
24 Jun
5:56 p.m.
On 23/06/2022 22:28, Andy Smith via BitFolk Users wrote:
Didn't take long for the first issue then!
When I tried to move a few mailing lists from Mailman2 to Mailman3,
there were plenty of issues before a single message went out.
Well more like all of them, really: I gave up and never did get Mailman3
to work!
--
Iain
23 Jun
23 Jun
10:28 p.m.
Thanks for doing this. I'm sure it wasn't much fun.
On Thu, Jun 23, 2022 at 09:16:00PM +0000, Andy Smith via BitFolk Users wrote:
=== Headers ===
I was confused by some of the headers on the message for a couple of
minutes. Because the changed List-ID defeated my filter making your
message land in my inbox, the followup "Oops" message arrived 7 minutes
before the "announce" one, and the message had these headers:
To: Iain Lane <iain(a)orangesquash.org.uk>
Cc: Andy Smith <andy(a)bitfolk.com>
...I was misled into thinking that you had mailed me directly, like
perhaps I'd expressed an interest in talking about mailing list
configuration in the past and you wanted to chat to me about it. I'm
used to looking at To to see where the poster sent the message to and
this made me think it was a direct mail.
Is there a reason for doing it like this, or could it be changed to the
'traditional' way?
(This is not hugely important. I'm sure I can adjust.)
=== Reply-To: added, no more "users-replyto"
===
[…]
It should also be noted that one of the objections to Reply-To:
munging is that it can remove all information about where the
author wants replies to go. In Mailman3's case, if you supply
Reply-To: headers of your own then it adds them, so that objection
doesn't hold.
Speaking for me personally, I'm happy with setting a default Reply-To
but I don't like the behaviour of Mailman 3 where it concatenates the
list's Reply-To with my supplied one. If I put a Reply-To on a list mail
it's because I want replies to go elsewhere and *not* to the list (or if
I do, I'll include the list's address myself). I wish this could be
changed but I'm not sure it can with the knobs Mailman 3 providers.
Cheers,
--
Iain Lane [ iain(a)orangesquash.org.uk ]
Debian Developer [ laney(a)debian.org ]
Ubuntu Developer [ laney(a)ubuntu.com ]
11:26 p.m.
HI Iain,
On Thu, Jun 23, 2022 at 11:28:10PM +0100, Iain Lane via BitFolk Users wrote:
the followup "Oops" message arrived 7
minutes
before the "announce" one
This was because Exim on the lists host was set to queue emails
after only 10 messages have been submitted in one connection - a
default setting I wasn't aware of and hadn't hit in my testing. The
queue check interval was also at its default, which is every 30
minutes.
So, a batch of emails went out, then there was a big delay before a
batch more were sent, and some later ones (which got immediately
sent, not queued), got delivered first. People who temporarily
rejected the emails (greylisting) also got a much bigger delay due
to multiple queue runs being needed.
I've now rectified that by removing the 10 message limit and
reducing the queue run frequency to 5 minutes. Thje re-ordering and
extreme delays shouldn't happen again.
the message had these headers:
To: Iain Lane <iain(a)orangesquash.org.uk>
Cc: Andy Smith <andy(a)bitfolk.com>
...I was misled into thinking that you had mailed me directly, like
perhaps I'd expressed an interest in talking about mailing list
configuration in the past and you wanted to chat to me about it. I'm
used to looking at To to see where the poster sent the message to and
this made me think it was a direct mail.
Is there a reason for doing it like this, or could it be changed to the
'traditional' way?
It has put your address in the To: because mails to this list
("users") are set to have full personalisation, i.e. some variable
expansions are available that relate to each individual subscriber.
The most useful of these is being able to put in the footer which
address you are subscribed under.
However, even though I thought I had set that (and it works on my
test list), that footer doesn't seem to have been used this time.
I'm unsure why and will have to look into that. It's supposed to look
like this:
_______________________________________________
$display_name mailing list <$listname>
You're subscribed as <$user_email>
Unsubscribe: <https://mailman.bitfolk.com/mailman/postorius/lists/$list_id/>
or send an email to <${short_listname}-leave@${domain}>
If that is deemed not particularly useful then I can turn off full
personalisation and the behaviour would go back to saying:
To: BitFolk Users <users(a)mailman.bitfolk.com>
Do people prefer that enough to get rid of the subscriber address in
footer?
It is quite common for people to forget under which email address
they have subscribed, and not know how to find that in the headers,
and then be unable to unsubscribe or log in to Mailman. They
sometimes end up trying to unsubscribe addresses that aren't
subscribed, or even subscribing again under a new address, thinking
that this is what is required.
Speaking for me personally, I'm happy with setting
a default Reply-To
but I don't like the behaviour of Mailman 3 where it concatenates the
list's Reply-To with my supplied one. If I put a Reply-To on a list mail
it's because I want replies to go elsewhere and *not* to the list (or if
I do, I'll include the list's address myself). I wish this could be
changed but I'm not sure it can with the knobs Mailman 3 providers.
Yeah, it can't. It's either "list doesn't add a Reply-To:" or
"list
adds its own to whatever is there already, if any".
Very few people were actually subscribed to "users-replyto" so it
could be argued that very few actually want the Reply-To: set, but
in conversation I actually think it's the case that most people were
just putting up with it not being set because they couldn't be
bothered to join the other list.
However, if there is a clear preference for not setting the
Reply-To: then I will revert that, and that would also allow any
user supplied Reply-To: to be the only one. So if you are not
particularly bothered either way about the list adding its own
Reply-To:, you might still be against it for the other reason. What
are your thoughts?
Thanks,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
24 Jun
24 Jun
12:04 a.m.
On Thu, Jun 23, 2022 at 11:26:12PM +0000, Andy Smith via BitFolk Users wrote:
It has put your address in the To: because mails to
this list
("users") are set to have full personalisation, i.e. some variable
expansions are available that relate to each individual subscriber.
The most useful of these is being able to put in the footer which
address you are subscribed under.
Okay, I worked out that the Personalize setting can be set to
"Individual" instead of "full", to still get that feature while
not messing with the To: header.
I think I may have fixed the footer issues also.
Cheers,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
25 Jun
25 Jun
9:53 a.m.
Hi Andy,
I just want to know about the Exim settings. My questions are below relevant
sections -
On Thu, Jun 23, 2022 at 11:26:12PM +0000, Andy Smith via BitFolk Users wrote:
HI Iain,
On Thu, Jun 23, 2022 at 11:28:10PM +0100, Iain Lane via BitFolk Users wrote:
What was the setting on Exim for "set to queue emails after only 10 messages
have been submitted in one connection"?
the followup "Oops" message arrived 7
minutes
before the "announce" one
This was because Exim on the lists host was set to queue emails
after only 10 messages have been submitted in one connection - a
default setting I wasn't aware of and hadn't hit in my testing. The
queue check interval was also at its default, which is every 30
minutes.
So, a batch of emails went out, then there was a big delay before a
batch more were sent, and some later ones (which got immediately
sent, not queued), got delivered first. People who temporarily
rejected the emails (greylisting) also got a much bigger delay due
to multiple queue runs being needed.
I've now rectified that by removing the 10 message limit and
reducing the queue run frequency to 5 minutes. Thje re-ordering and
extreme delays shouldn't happen again.
...and the settings that you now use to rectify?
the message had these headers:
To: Iain Lane <iain(a)orangesquash.org.uk>
Cc: Andy Smith <andy(a)bitfolk.com>
...I was misled into thinking that you had mailed me directly, like
perhaps I'd expressed an interest in talking about mailing list
configuration in the past and you wanted to chat to me about it. I'm
used to looking at To to see where the poster sent the message to and
this made me think it was a direct mail.
Is there a reason for doing it like this, or could it be changed to the
'traditional' way?
It has put your address in the To: because mails to this list
("users") are set to have full personalisation, i.e. some variable
expansions are available that relate to each individual subscriber.
The most useful of these is being able to put in the footer which
address you are subscribed under.
However, even though I thought I had set that (and it works on my
test list), that footer doesn't seem to have been used this time.
I'm unsure why and will have to look into that. It's supposed to look
like this:
_______________________________________________
$display_name mailing list <$listname>
You're subscribed as <$user_email>
Unsubscribe: <https://mailman.bitfolk.com/mailman/postorius/lists/$list_id/>
or send an email to <${short_listname}-leave@${domain}>
If that is deemed not particularly useful then I can turn off full
personalisation and the behaviour would go back to saying:
To: BitFolk Users <users(a)mailman.bitfolk.com>
Do people prefer that enough to get rid of the subscriber address in
footer?
It is quite common for people to forget under which email address
they have subscribed, and not know how to find that in the headers,
and then be unable to unsubscribe or log in to Mailman. They
sometimes end up trying to unsubscribe addresses that aren't
subscribed, or even subscribing again under a new address, thinking
that this is what is required.
Speaking for me personally, I'm happy with
setting a default Reply-To
but I don't like the behaviour of Mailman 3 where it concatenates the
list's Reply-To with my supplied one. If I put a Reply-To on a list mail
it's because I want replies to go elsewhere and *not* to the list (or if
I do, I'll include the list's address myself). I wish this could be
changed but I'm not sure it can with the knobs Mailman 3 providers.
Yeah, it can't. It's either "list doesn't add a Reply-To:" or
"list
adds its own to whatever is there already, if any".
Very few people were actually subscribed to "users-replyto" so it
could be argued that very few actually want the Reply-To: set, but
in conversation I actually think it's the case that most people were
just putting up with it not being set because they couldn't be
bothered to join the other list.
However, if there is a clear preference for not setting the
Reply-To: then I will revert that, and that would also allow any
user supplied Reply-To: to be the only one. So if you are not
particularly bothered either way about the list adding its own
Reply-To:, you might still be against it for the other reason. What
are your thoughts?
Thanks,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting _______________________________________________
BitFolk Users mailing list -- users(a)mailman.bitfolk.com
To unsubscribe send an email to users-leave(a)mailman.bitfolk.com
--
Regards,
Didar
<knghtbrd> rcw: Oh yay---I haven't been involved in a good flamewar in at
least ... 5 minutes!
Generated by Signify v1.14 (http://www.debian.org/)
12:24 p.m.
Hi Didar,
On Sat, Jun 25, 2022 at 03:23:54PM +0530, didar via BitFolk Users wrote:
I set smtp_accept_queue_per_connection to 0 so it will allow any
number of messages in a connection.
Just in case there should be some runaway deluge of email I also
added:
queue_only_load = 4
so that if system load goes >= 4 it will queue mails, not try to
deliver them immediately.
On Debian the main Exim process also does queue runs, and the
interval is set in /etc/default/exim4:
QUEUEINTERVAL='5m'
(default is '30m')
Cheer,s
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
On Thu, Jun 23, 2022 at 11:26:12PM +0000, Andy Smith
via BitFolk Users wrote:
The setting is smtp_accept_queue_per_connection. On Debian's Exim
this isn't set, and the default value is 10.
https://www.exim.org/exim-html-current/doc/html/spec_html/ch-main_configura…
"This option limits the number of delivery processes that Exim
starts automatically when receiving messages via SMTP, whether
via the daemon or by the use of -bs or -bS. If the value of the
option is greater than zero, and the number of messages received
in a single SMTP session exceeds this number, subsequent
messages are placed in the queue, but no delivery processes are
started. This helps to limit the number of Exim processes when a
server restarts after downtime and there is a lot of mail
waiting for it on other systems. On large systems, the default
should probably be increased, and on dial-in client systems it
should probably be set to zero (that is, disabled)."
This was because Exim on the lists host was set
to queue emails
after only 10 messages have been submitted in one connection - a
default setting I wasn't aware of and hadn't hit in my testing. The
queue check interval was also at its default, which is every 30
minutes.
What was the setting on Exim for "set to queue emails after only 10 messages
have been submitted in one connection"? I've now
rectified that by removing the 10 message limit and
reducing the queue run frequency to 5 minutes. Thje re-ordering and
extreme delays shouldn't happen again.
...and the settings that you now use to rectify?
29 Jun
29 Jun
10:46 a.m.
On Thu, Jun 23, 2022 at 11:26:12PM +0000, Andy Smith via BitFolk Users wrote:
I agree it's likely to be a "default effect" that few users subscribed
to the alternate list and I'm happy with what you've got set up now. My
grumble is with Mailman upstream for not letting me just override the
Reply-To with my own values when I want to. Probably not a problem for
this list really. I do that sometimes for Debian but they don't use
Mailman anyway. :-)
Cheers,
--
Iain Lane [ iain(a)orangesquash.org.uk ]
Debian Developer [ laney(a)debian.org ]
Ubuntu Developer [ laney(a)ubuntu.com ]
However, even though I thought I had set that (and it
works on my
test list), that footer doesn't seem to have been used this time.
I'm unsure why and will have to look into that. It's supposed to look
like this:
_______________________________________________
$display_name mailing list <$listname>
You're subscribed as <$user_email>
Unsubscribe: <https://mailman.bitfolk.com/mailman/postorius/lists/$list_id/>
or send an email to <${short_listname}-leave@${domain}>
If that is deemed not particularly useful then I can turn off full
personalisation and the behaviour would go back to saying:
To: BitFolk Users <users(a)mailman.bitfolk.com>
Do people prefer that enough to get rid of the subscriber address in
footer?
I think what you have now (To: list and a customised footer) is great.
Thanks!
Speaking for
me personally, I'm happy with setting a default
Reply-To but I don't like the behaviour of Mailman 3 where it
concatenates the list's Reply-To with my supplied one. If I put a
Reply-To on a list mail it's because I want replies to go elsewhere
and *not* to the list (or if I do, I'll include the list's address
myself). I wish this could be changed but I'm not sure it can with
the knobs Mailman 3 providers.
Yeah, it can't. It's either "list doesn't add a Reply-To:" or
"list
adds its own to whatever is there already, if any".
Very few people were actually subscribed to "users-replyto" so it
could be argued that very few actually want the Reply-To: set, but
in conversation I actually think it's the case that most people were
just putting up with it not being set because they couldn't be
bothered to join the other list.
However, if there is a clear preference for not setting the
Reply-To: then I will revert that, and that would also allow any
user supplied Reply-To: to be the only one. So if you are not
particularly bothered either way about the list adding its own
Reply-To:, you might still be against it for the other reason. What
are your thoughts?
2 Jul
2 Jul
7:30 a.m.
On Thu, Jun 23, 2022 at 09:16:00PM +0000, Andy Smith via BitFolk Users wrote:
=== List archives ===
I hope to be able to transplant a static copy of
https://lists.bitfolk.com/lurker/ over to
https://mailman.bitfolk.com/lurker/ so that old links will still
work, with automatic redirection.
This is now done, so old links under
https://lists.bitfolk.com/lurker/… redirect to the same place under
https://mailman.bitfolk.com/lurker/, which work, so at least we were
able to preserve existing archive links out there in the wild.
Probably still best to update any links you find in BitFolk
documentation etc.
I also redirected the root of https://(lists|mailman).bitfolk.com/
to the landing page at
https://mailman.bitfolk.com/mailman/postorius/lists/
Cheers,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
664
days inactive
673
days old
9 comments
4 participants
participants (4)
-
Andy Smith -
didar -
Iain Harrison -
Iain Lane