Category Archives: Email stuff

Mutt: delete duplicate e-mail messages

Disclaimer: at some point I found this page describing how to delete duplicate e-mail messages with mutt. Unfortunately the page is not up anymore (HTTP 404) and I took the liberty to post it on my blog. Anyway, credit goes to Marianne Promberger. The original page was here: http://promberger.info/linux/2008/03/31/mutt-delete-duplicate-e-mail-messages/

Here we go:

Sometimes, if you consolidate different mailboxes where some of the messages are in both mailboxes, you end up with duplicates.
With mutt, it is really easy to delete one copy of each duplicate. I got this tip from here.

You need to have set duplicate_threads = yes, either put it in your~/.muttrc, or check whether it is on by default (it is for me). To see the value of a variable, while you’re running mutt, say

:set ?duplicate_threads

and it will display the current value (note the leading colon, and of course replace “duplicate_threads” with the variable name you want to see).

You also need to have your mailbox sorted by message threads. This is a nice feature in general, similar to the messages threads in Gmail (but mutt’s implementation is much more user friendly, in my opinion). If you haven’t alreadyset sort=threads in your ~/.muttrc, you can sort “on the fly” while you’re in a mailbox: type o (to sort; mutt then asks for the criterion to sort on, and tells you the options) followed by d (for date).

Now say T to tag a certain pattern, put in ~= as the pattern. Duplicates (one copy of each message that mutt sees twice in the folder) are now tagged. To delete them, either type just d (this will work if you have set auto_tag=yes), or type; to apply the next command to all tagged messages, then hit d.

Addendum

Actually, it’s much easier, you can skip the tagging step and just do D (for “delete matching pattern” followed by ~=. If you’re adventurous, you can set Mutt to automatically rid your mailboxes of duplicates using a folder-hook, like this (in your ~/.muttrc)

folder-hook . push "<delete-pattern>~=<enter>"

This is handy together with

folder-hook . 'set record="^"'

which always puts your “sent” copy into the current mailbox — handy for developing meaningful threads of incoming mails and replies. However, of course it results in duplicates if you are on the cc or if the mail is going to a mailing list you’re subscribed to. Enter the above folder-hook — no more duplicates.

If you’re even more adventurous, you could add updating your mailbox (i.e., purging messages marked for deletion) to the above folder-hook, like this:

folder-hook . push "<delete-pattern>~=<enter>$"

Migrate your IMAP account to Microsoft Office 365

What will you do if your organization decided to move email, calendar and so on into the cloud and they choose Microsoft Office 365 and your task is to move all emails from your current server into cloud. How do you do to complete this task? First of all you search on the web (as I did) and you will find that there are some tools and documents. For me choosing a tool was the difficult part, but at the end I decided that imapsync is the best tool for this kind of tasks.

Ok, let's finish this small talk and let's get to work.

1. Get imapsync

[codesyntax lang="bash"]

git clone https://github.com/imapsync/imapsync

[/codesyntax]

2. Installing required packages

[codesyntax lang="bash"]

apt-get install libmail-imapclient-perl libterm-readkey-perl

[/codesyntax]

3. Checking dependencies

[codesyntax lang="bash"]

cd imapsync
perl -c imapsync

[/codesyntax]

And if nothing is missing then you will see a message like this:

imapsync syntax OK

4. Create the password files

[codesyntax lang="bash"]

echo "secret_password1" > ~/secret1
echo "secret_password2" > ~/secret2
chmod 600 ~/secret1
chmod 600 ~/secret2

[/codesyntax]

5. Do the migration

[codesyntax lang="bash"]

./imapsync --nocheckmessageexists --syncinternaldates --usecache --useuid --host1 example.com --user1 "user@example.com" --passfile ~/secret1 --ssl1 --host2 pod51011.outlook.com --user2 'user@example.com' --passfile ~/secret2 --ssl2

[/codesyntax]

example.com - is the host from where you want to pull the mails
pod51011.outlook.com - is the microsoft server where you want to push the mails

Note: if you already executed the above command before then you should use --maxage n parameter so imapsync to check and pull and push only mail no more older than n days