View previous topic :: View next topic |
Author |
Message |
Il turisto l33t


Joined: 12 May 2004 Posts: 968 Location: Battincourt - Belgique
|
Posted: Wed Jan 04, 2006 9:09 am Post subject: Backup of old mail |
|
|
Hi,
I have a server mail (exim) with a big packet of mail (more than 100 gigas).
My request is simple in fact. I wanna know if someone have a script who take the mail older than a date, make an archive an delete it from the file system.
Of course the directory structure must rest the same because this older mail are sometimes useful.
Thanks for help. |
|
Back to top |
|
 |
Il turisto l33t


Joined: 12 May 2004 Posts: 968 Location: Battincourt - Belgique
|
Posted: Thu Jan 05, 2006 11:12 am Post subject: |
|
|
no one can help me? |
|
Back to top |
|
 |
nevynxxx Veteran

Joined: 12 Nov 2003 Posts: 1123 Location: Manchester - UK
|
Posted: Thu Jan 05, 2006 12:46 pm Post subject: |
|
|
Is it maildir or mbox?
If it's maildir, any backup too can be used as each mail is a sperate file, so the backup script can find them individually and move them.
I do some automatic filtering on the server side for my mail boxes like this.
If it's mbox you will probably need some special software I think.
Tried the exim users mailing list? _________________ My Public Key
Wanted: Instructor in the art of Bowyery |
|
Back to top |
|
 |
Il turisto l33t


Joined: 12 May 2004 Posts: 968 Location: Battincourt - Belgique
|
Posted: Thu Jan 05, 2006 12:55 pm Post subject: |
|
|
It's maildir.
I have a file per mail.
Can you show me your script? |
|
Back to top |
|
 |
Proteus Guru


Joined: 14 Jul 2002 Posts: 346 Location: Hamburg, Germany
|
Posted: Thu Jan 05, 2006 1:58 pm Post subject: |
|
|
I don't know about a script for suc a task but my idea would be to setup an Imap server,
connect to it with a client and then download all the mails you want to backup and backup your
complete imap client.
But I am sure that a script-guru would do it much faster and better Just an idea. _________________ Greetings,
Proteus |
|
Back to top |
|
 |
nevynxxx Veteran

Joined: 12 Nov 2003 Posts: 1123 Location: Manchester - UK
|
Posted: Thu Jan 05, 2006 3:01 pm Post subject: |
|
|
Il turisto wrote: | It's maildir.
I have a file per mail.
Can you show me your script? |
not at the moment, my serevr is down
try man find, if you're archiving old mail.
It can find all files created after a specific date, under a specific dir. find can also run a command on each file so something like
find <path to mail store> -mtime date -exec mv {} backup/mail/;
should do the trick, then tar that up and store it..... _________________ My Public Key
Wanted: Instructor in the art of Bowyery |
|
Back to top |
|
 |
adsmith Veteran

Joined: 26 Sep 2004 Posts: 1386 Location: NC, USA
|
Posted: Thu Jan 05, 2006 6:54 pm Post subject: |
|
|
emerge archivemail, a beautiful python script which does everything you want and more. |
|
Back to top |
|
 |
Il turisto l33t


Joined: 12 May 2004 Posts: 968 Location: Battincourt - Belgique
|
Posted: Fri Jan 06, 2006 8:33 am Post subject: |
|
|
@Proteus : it is an imap server but I have a lot of mailboxes.
@nevyxxx : sorry for your server. I hope that this is not too important.
I have tried the command find but it lost the directory structure.
@adsmith : Great idea I wil try that and post my comment here. I think that this script is what I need. Does it delete backuped mail or I have to delete it myself? (Bad question I will try ) |
|
Back to top |
|
 |
Il turisto l33t


Joined: 12 May 2004 Posts: 968 Location: Battincourt - Belgique
|
Posted: Fri Jan 06, 2006 9:44 am Post subject: |
|
|
Ok guys backup work correctly but now how can I restore mails?
The soft make a gz files. When I gunzip this file I have a big text file. and of course I can't use it with my mail server.
edit : this soft only archive cur directory. If the user have made some subdirectory they are not backuped. A solution? |
|
Back to top |
|
 |
adsmith Veteran

Joined: 26 Sep 2004 Posts: 1386 Location: NC, USA
|
Posted: Fri Jan 06, 2006 2:50 pm Post subject: |
|
|
Well, you can run it on multiple directories -- a simple "find" script will do that for you.
I run
Code: |
archivemail -d 90 -s '' -o ~/ARCHIVES/ ~/mail/*
|
in my crontab, but you could easily do something like:
Code: |
find ~/mail/* -type d | while read thisdir; do
archivemail -d 90 -s '' -o "~/ARCHIVES/$(basename thisdir)" "$thisdir"
done
|
NOTE: I have not sanity checked this code on real data!
You can easily disable the gzipping if you don't want it.
However, a clever mail client like mutt will read gzipped mailboxes just fine. Sadly, this doesn't work through IMAP, only locally. |
|
Back to top |
|
 |
Il turisto l33t


Joined: 12 May 2004 Posts: 968 Location: Battincourt - Belgique
|
Posted: Mon Jan 09, 2006 12:57 pm Post subject: |
|
|
thansk for the script I will try it.
But archivemail made one file with all mails into it and I have one file per mail in my filesystem.
So how can I restore mail? |
|
Back to top |
|
 |
adsmith Veteran

Joined: 26 Sep 2004 Posts: 1386 Location: NC, USA
|
Posted: Mon Jan 09, 2006 2:32 pm Post subject: |
|
|
archivemail should give you one archive file per mailbox, which is the logical way to do it, I think.
Do you really want one archive file per message, or are you saying that it somehow gave you one archive file for *all* mailboxes?
As I said, I haven't used it on maildir's, but in any case, the default output format of archivemail is a gzipped mbox. So, gunzip it to a flat mbox file, then use any mbox->maildir converter to restore all the mail. |
|
Back to top |
|
 |
Il turisto l33t


Joined: 12 May 2004 Posts: 968 Location: Battincourt - Belgique
|
Posted: Tue Jan 10, 2006 7:49 am Post subject: |
|
|
Yes it gave me one archive per mailbox and this is perfect.
But in this archive it put all the mails in one big text file. And I have mailbox type so I have a file per mail.
So when I deflatte the archive I cannot use the big file for restore my mails. |
|
Back to top |
|
 |
adsmith Veteran

Joined: 26 Sep 2004 Posts: 1386 Location: NC, USA
|
Posted: Tue Jan 10, 2006 12:58 pm Post subject: |
|
|
Right, so you just need any mbox -> maildir utility to convert that back to maildir. These programs are all over the place if you check google, etc.
(This sort of problem is exactly why I prefer mbox to begin with. It's just so darn simple, and you can fix any mailbox error with your favorite text editor.) |
|
Back to top |
|
 |
Il turisto l33t


Joined: 12 May 2004 Posts: 968 Location: Battincourt - Belgique
|
Posted: Tue Jan 10, 2006 2:28 pm Post subject: |
|
|
The problem is a little more complicated because if I have to restore mail it is one or two and not all.
Thanks for your reponse |
|
Back to top |
|
 |
adsmith Veteran

Joined: 26 Sep 2004 Posts: 1386 Location: NC, USA
|
Posted: Tue Jan 10, 2006 2:41 pm Post subject: |
|
|
Well, one (craptastic) way to do this is to use a mail client which reads both mbox and maildir, then move the messages from the mbox mailbox to a maildir mailbox.
alternatively, just go back to writing a custom bash script in a cron job..... |
|
Back to top |
|
 |
BitJam Advocate

Joined: 12 Aug 2003 Posts: 2513 Location: Silver City, NM
|
Posted: Thu Mar 09, 2006 7:37 am Post subject: |
|
|
Il turisto,
I just wrote a Perl program that archives mail in maildirs to maildirs under a new directory tree. Here is the usage: Code: | Usage: archive-maildir [options] maildir [new_maildir]
Options:
-d --dirs Read directory list from STDIN
-h --help Print this help.
-l --list Just list the directories found.
-m --mkdir Make destination dirs even if empty.
-n --newer DAYS Moves files newer than DAYS
-o --older DAYS Moves files older than DAYS
-p --pretend Don't really move anything.
-v --verbose Print what we do as we do it.
You can specify both --older and --newer.
If a new_maildir is not specified, maildir.archive is used. |
Here is a sample output (in pretend mode): Code: | ./archive-maildir --pretend --older 800 Mail/
Move (25) files to Mail.archive/Misc/cur
Move (16) files to Mail.archive/.Misc.directory/EarthLink/cur
Move (19) files to Mail.archive/drafts/cur
Move (1143) files to Mail.archive/sent-mail/cur
Move (15) files to Mail.archive/.Friends.directory/Nia/cur
Move (33) files to Mail.archive/.Friends.directory/Emma/cur
Move (365) files to Mail.archive/.Friends.directory/Alvin/cur |
It looks for cur, new, and tmp directories but by default only makes new directories as needed which is why tmp and new don't show up in the output. It shells out to the find command for finding the directories and then finding the files within the directories which can make it a tad slow for small directories but I hope it will make it robust for larger problems such as yours.
The initial find (for all the cur, tmp and new) directories can be a bit slow if you want to play around in pretend mode. That's why I added the --dirs and --list options. They allow you to cache the list of directories in a file: Code: | $ archive-maildir --list Mail/ > dirs.txt
$ cat dirs.txt | archive-maildir --dirs --pretend --older $ODAYS Mail/ |
You can also put in greps after the cat to only deal with a subset of the directories at a wack.
I'll post the code if you are interested. |
|
Back to top |
|
 |
carpman Advocate

Joined: 20 Jun 2002 Posts: 2202 Location: London - UK
|
Posted: Sat May 26, 2007 2:07 pm Post subject: |
|
|
Hello, been trying to get this working but having 2 problems:
Working on multiple user mailboxs uses one command
using suggestion earlier in thread
Code: |
archivemail -n -d360 -s '' -o /home/mail-bak ~/.maildir/*
archivemail: '/root/.maildir/.*': no such file or directory
|
Of course i have no .maildir in root so would need to excluded that
Any suggestion getting this to work? _________________ Work Station - 64bit
Gigabyte GA X48-DQ6 Core2duo E8400
8GB GSkill DDR2-1066
SATA Areca 1210 Raid
BFG OC2 8800 GTS 640mb
--------------------------------
Notebook
Samsung Q45 7100 4gb |
|
Back to top |
|
 |
adsmith Veteran

Joined: 26 Sep 2004 Posts: 1386 Location: NC, USA
|
Posted: Sat May 26, 2007 2:22 pm Post subject: |
|
|
Are you sure "~/.maildir/*" is correct? What about just "~/.maildir" or "~/.maildir/.*"?
In the Maildir standard, any sub-folders are named with leading dots, so * would not catch them.
E.g., under ~/.maildir you may have
.folder
.folder.subfolder
courierimapkeywords
archivemail only sees that one file with the glob you use, but this is a just a dumb cache file and not a mail folder. |
|
Back to top |
|
 |
carpman Advocate

Joined: 20 Jun 2002 Posts: 2202 Location: London - UK
|
Posted: Sat May 26, 2007 2:23 pm Post subject: |
|
|
Ok realised that i should have /.* so tried again with:
Code: |
archivemail -n -d360 -s '' -o /home/mail-bak ~/.maildir/.*
|
Also tried after create dir /root/.maildir
Code: |
archivemail -n -d360 -s '' -o /home/mail-bak ~/.maildir/.*
/root/.maildir/.: I would have archived 0 of 0 message(s) in 0.0 seconds
/root/.maildir/..: I would have archived 0 of 0 message(s) in 0.0 seconds
|
_________________ Work Station - 64bit
Gigabyte GA X48-DQ6 Core2duo E8400
8GB GSkill DDR2-1066
SATA Areca 1210 Raid
BFG OC2 8800 GTS 640mb
--------------------------------
Notebook
Samsung Q45 7100 4gb |
|
Back to top |
|
 |
adsmith Veteran

Joined: 26 Sep 2004 Posts: 1386 Location: NC, USA
|
Posted: Sat May 26, 2007 2:35 pm Post subject: |
|
|
uhh.. so what's actually there?
what's the output of
"find ~/.maildir -maxdepth 1" |
|
Back to top |
|
 |
carpman Advocate

Joined: 20 Jun 2002 Posts: 2202 Location: London - UK
|
Posted: Sat May 26, 2007 3:01 pm Post subject: |
|
|
adsmith wrote: | uhh.. so what's actually there?
what's the output of
"find ~/.maildir -maxdepth 1" |
Out put is:
Code: |
find ~/.maildir -maxdepth 1
find: /root/.maildir: No such file or directory
|
It is just looking in dir of root all the time?
There are 3 users on system root and one user and one for sysadmin, root does not have .maildir and does not need one.
Both sysadmin and user do have .maildir though only user has mail old enough to need archiving at moment. If i run archivemail on the user only it works fine:
Code: |
archivemail -n -d465 -s '' -o /home/mail-bak /home/sara/.maildir/.*
/home/sara/.maildir/.: I would have archived 178 of 865 message(s) in 2.7 seconds
/home/sara/.maildir/..: I would have archived 0 of 0 message(s) in 0.0 seconds
/home/sara/.maildir/.Clients: I would have archived 0 of 0 message(s) in 0.0 seconds
/home/sara/.maildir/.Drafts: I would have archived 43 of 143 message(s) in 0.3 seconds
/home/sara/.maildir/.Leads: I would have archived 0 of 0 message(s) in 0.0 seconds
/home/sara/.maildir/.Personal: I would have archived 30 of 205 message(s) in 0.6 seconds
/home/sara/.maildir/.Sales: I would have archived 0 of 0 message(s) in 0.0 seconds
/home/sara/.maildir/.Sent: I would have archived 768 of 2762 message(s) in 4.6 seconds
/home/sara/.maildir/.Pub: I would have archived 110 of 692 message(s) in 2.0 seconds
/home/sara/.maildir/.Support: I would have archived 0 of 0 message(s) in 0.0 seconds
/home/sara/.maildir/.Trash: I would have archived 0 of 1 message(s) in 0.0 seconds
/home/sara/.maildir/.VirusAlerts: I would have archived 0 of 0 message(s) in 0.0 seconds
/home/sara/.maildir/.spam: I would have archived 14 of 68 message(s) in 0.2 seconds
|
It just won't do this if try running on multiple user .maildir ?
.maildir is there for users:
Code: |
ls -la /home/sara_davis/
total 1275340
drwxr-xr-x 7 sara sara 4096 May 24 02:46 .
drwxr-xr-x 7 root root 4096 May 26 14:35 ..
-rw-r--r-- 1 sara sara 127 May 23 14:46 .bash_logout
-rw-r--r-- 1 sara sara 193 May 23 14:46 .bash_profile
-rw-r--r-- 1 sara sara 551 May 23 14:46 .bashrc
drwx------ 18 sara sara 4096 May 26 15:12 .maildir
|
_________________ Work Station - 64bit
Gigabyte GA X48-DQ6 Core2duo E8400
8GB GSkill DDR2-1066
SATA Areca 1210 Raid
BFG OC2 8800 GTS 640mb
--------------------------------
Notebook
Samsung Q45 7100 4gb |
|
Back to top |
|
 |
adsmith Veteran

Joined: 26 Sep 2004 Posts: 1386 Location: NC, USA
|
Posted: Sun May 27, 2007 12:25 pm Post subject: |
|
|
Yes, of course. ~ expands to $HOME of the *current* user. If you want to do this as a cron job, it must be the user's cron job (or you muck aroundwith sudo in your cron script). |
|
Back to top |
|
 |
carpman Advocate

Joined: 20 Jun 2002 Posts: 2202 Location: London - UK
|
Posted: Sun May 27, 2007 4:28 pm Post subject: |
|
|
adsmith wrote: | Yes, of course. ~ expands to $HOME of the *current* user. If you want to do this as a cron job, it must be the user's cron job (or you muck aroundwith sudo in your cron script). |
Ok but as i am system admin i need to run this on all users mail and want to do it with one cronjob not individual cron jobs for each user? _________________ Work Station - 64bit
Gigabyte GA X48-DQ6 Core2duo E8400
8GB GSkill DDR2-1066
SATA Areca 1210 Raid
BFG OC2 8800 GTS 640mb
--------------------------------
Notebook
Samsung Q45 7100 4gb |
|
Back to top |
|
 |
adsmith Veteran

Joined: 26 Sep 2004 Posts: 1386 Location: NC, USA
|
Posted: Sun May 27, 2007 6:49 pm Post subject: |
|
|
OK, this is no longer an archivemail problem, but a combination of bash and cron.
First, you should go google for "Advanced bash scripting guide" for some pointers on scripting.
some cobination of archivemail, chown, and maybe sudo will do what you want, I think.
Although... I'm not sure why you want root to cycle other user's mailboxes, since users generally want complete control of their own mailboxes. The users could always run their own cron job... |
|
Back to top |
|
 |
|