Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Backup of old mail
View unanswered posts
View posts from last 24 hours

Goto page 1, 2  Next  
Reply to topic    Gentoo Forums Forum Index Networking & Security
View previous topic :: View next topic  
Author Message
Il turisto
l33t
l33t


Joined: 12 May 2004
Posts: 968
Location: Battincourt - Belgique

PostPosted: Wed Jan 04, 2006 9:09 am    Post subject: Backup of old mail Reply with quote

Hi,

I have a server mail (exim) with a big packet of mail (more than 100 gigas).

My request is simple in fact. I wanna know if someone have a script who take the mail older than a date, make an archive an delete it from the file system.

Of course the directory structure must rest the same because this older mail are sometimes useful.


Thanks for help.
Back to top
View user's profile Send private message
Il turisto
l33t
l33t


Joined: 12 May 2004
Posts: 968
Location: Battincourt - Belgique

PostPosted: Thu Jan 05, 2006 11:12 am    Post subject: Reply with quote

no one can help me?
Back to top
View user's profile Send private message
nevynxxx
Veteran
Veteran


Joined: 12 Nov 2003
Posts: 1123
Location: Manchester - UK

PostPosted: Thu Jan 05, 2006 12:46 pm    Post subject: Reply with quote

Is it maildir or mbox?

If it's maildir, any backup too can be used as each mail is a sperate file, so the backup script can find them individually and move them.

I do some automatic filtering on the server side for my mail boxes like this.

If it's mbox you will probably need some special software I think.

Tried the exim users mailing list?
_________________
My Public Key

Wanted: Instructor in the art of Bowyery
Back to top
View user's profile Send private message
Il turisto
l33t
l33t


Joined: 12 May 2004
Posts: 968
Location: Battincourt - Belgique

PostPosted: Thu Jan 05, 2006 12:55 pm    Post subject: Reply with quote

It's maildir.

I have a file per mail.

Can you show me your script?
Back to top
View user's profile Send private message
Proteus
Guru
Guru


Joined: 14 Jul 2002
Posts: 346
Location: Hamburg, Germany

PostPosted: Thu Jan 05, 2006 1:58 pm    Post subject: Reply with quote

I don't know about a script for suc a task but my idea would be to setup an Imap server,
connect to it with a client and then download all the mails you want to backup and backup your
complete imap client.

But I am sure that a script-guru would do it much faster and better :-) Just an idea.
_________________
Greetings,
Proteus
Back to top
View user's profile Send private message
nevynxxx
Veteran
Veteran


Joined: 12 Nov 2003
Posts: 1123
Location: Manchester - UK

PostPosted: Thu Jan 05, 2006 3:01 pm    Post subject: Reply with quote

Il turisto wrote:
It's maildir.

I have a file per mail.

Can you show me your script?


not at the moment, my serevr is down :)

try man find, if you're archiving old mail.

It can find all files created after a specific date, under a specific dir. find can also run a command on each file so something like

find <path to mail store> -mtime date -exec mv {} backup/mail/;

should do the trick, then tar that up and store it.....
_________________
My Public Key

Wanted: Instructor in the art of Bowyery
Back to top
View user's profile Send private message
adsmith
Veteran
Veteran


Joined: 26 Sep 2004
Posts: 1386
Location: NC, USA

PostPosted: Thu Jan 05, 2006 6:54 pm    Post subject: Reply with quote

emerge archivemail, a beautiful python script which does everything you want and more.
Back to top
View user's profile Send private message
Il turisto
l33t
l33t


Joined: 12 May 2004
Posts: 968
Location: Battincourt - Belgique

PostPosted: Fri Jan 06, 2006 8:33 am    Post subject: Reply with quote

@Proteus : it is an imap server :-) but I have a lot of mailboxes.

@nevyxxx : sorry for your server. I hope that this is not too important.
I have tried the command find but it lost the directory structure.

@adsmith : Great idea I wil try that and post my comment here. I think that this script is what I need. Does it delete backuped mail or I have to delete it myself? (Bad question I will try ;-) )
Back to top
View user's profile Send private message
Il turisto
l33t
l33t


Joined: 12 May 2004
Posts: 968
Location: Battincourt - Belgique

PostPosted: Fri Jan 06, 2006 9:44 am    Post subject: Reply with quote

Ok guys backup work correctly but now how can I restore mails?

The soft make a gz files. When I gunzip this file I have a big text file. and of course I can't use it with my mail server.


edit : this soft only archive cur directory. If the user have made some subdirectory they are not backuped. A solution?
Back to top
View user's profile Send private message
adsmith
Veteran
Veteran


Joined: 26 Sep 2004
Posts: 1386
Location: NC, USA

PostPosted: Fri Jan 06, 2006 2:50 pm    Post subject: Reply with quote

Well, you can run it on multiple directories -- a simple "find" script will do that for you.

I run
Code:

archivemail -d 90 -s '' -o ~/ARCHIVES/ ~/mail/*

in my crontab, but you could easily do something like:
Code:

find ~/mail/* -type d | while read thisdir; do
   archivemail -d 90 -s '' -o "~/ARCHIVES/$(basename thisdir)" "$thisdir"
done

NOTE: I have not sanity checked this code on real data!

You can easily disable the gzipping if you don't want it.

However, a clever mail client like mutt will read gzipped mailboxes just fine. Sadly, this doesn't work through IMAP, only locally.
Back to top
View user's profile Send private message
Il turisto
l33t
l33t


Joined: 12 May 2004
Posts: 968
Location: Battincourt - Belgique

PostPosted: Mon Jan 09, 2006 12:57 pm    Post subject: Reply with quote

thansk for the script I will try it.

But archivemail made one file with all mails into it and I have one file per mail in my filesystem.

So how can I restore mail?
Back to top
View user's profile Send private message
adsmith
Veteran
Veteran


Joined: 26 Sep 2004
Posts: 1386
Location: NC, USA

PostPosted: Mon Jan 09, 2006 2:32 pm    Post subject: Reply with quote

archivemail should give you one archive file per mailbox, which is the logical way to do it, I think.
Do you really want one archive file per message, or are you saying that it somehow gave you one archive file for *all* mailboxes?

As I said, I haven't used it on maildir's, but in any case, the default output format of archivemail is a gzipped mbox. So, gunzip it to a flat mbox file, then use any mbox->maildir converter to restore all the mail.
Back to top
View user's profile Send private message
Il turisto
l33t
l33t


Joined: 12 May 2004
Posts: 968
Location: Battincourt - Belgique

PostPosted: Tue Jan 10, 2006 7:49 am    Post subject: Reply with quote

Yes it gave me one archive per mailbox and this is perfect.

But in this archive it put all the mails in one big text file. And I have mailbox type so I have a file per mail.

So when I deflatte the archive I cannot use the big file for restore my mails.
Back to top
View user's profile Send private message
adsmith
Veteran
Veteran


Joined: 26 Sep 2004
Posts: 1386
Location: NC, USA

PostPosted: Tue Jan 10, 2006 12:58 pm    Post subject: Reply with quote

Right, so you just need any mbox -> maildir utility to convert that back to maildir. These programs are all over the place if you check google, etc.

(This sort of problem is exactly why I prefer mbox to begin with. It's just so darn simple, and you can fix any mailbox error with your favorite text editor.)
Back to top
View user's profile Send private message
Il turisto
l33t
l33t


Joined: 12 May 2004
Posts: 968
Location: Battincourt - Belgique

PostPosted: Tue Jan 10, 2006 2:28 pm    Post subject: Reply with quote

The problem is a little more complicated because if I have to restore mail it is one or two and not all.


Thanks for your reponse
Back to top
View user's profile Send private message
adsmith
Veteran
Veteran


Joined: 26 Sep 2004
Posts: 1386
Location: NC, USA

PostPosted: Tue Jan 10, 2006 2:41 pm    Post subject: Reply with quote

Well, one (craptastic) way to do this is to use a mail client which reads both mbox and maildir, then move the messages from the mbox mailbox to a maildir mailbox.


alternatively, just go back to writing a custom bash script in a cron job.....
Back to top
View user's profile Send private message
BitJam
Advocate
Advocate


Joined: 12 Aug 2003
Posts: 2513
Location: Silver City, NM

PostPosted: Thu Mar 09, 2006 7:37 am    Post subject: Reply with quote

Il turisto,

I just wrote a Perl program that archives mail in maildirs to maildirs under a new directory tree. Here is the usage:
Code:
Usage:  archive-maildir [options] maildir [new_maildir]

Options:
        -d --dirs       Read directory list from STDIN
        -h --help       Print this help.
        -l --list       Just list the directories found.
        -m --mkdir      Make destination dirs even if empty.
        -n --newer DAYS Moves files newer than DAYS
        -o --older DAYS Moves files older than DAYS
        -p --pretend    Don't really move anything.
        -v --verbose    Print what we do as we do it.

You can specify both --older and --newer.

If a new_maildir is not specified, maildir.archive is used.


Here is a sample output (in pretend mode):
Code:
./archive-maildir  --pretend --older 800 Mail/
Move (25) files to Mail.archive/Misc/cur
Move (16) files to Mail.archive/.Misc.directory/EarthLink/cur
Move (19) files to Mail.archive/drafts/cur
Move (1143) files to Mail.archive/sent-mail/cur
Move (15) files to Mail.archive/.Friends.directory/Nia/cur
Move (33) files to Mail.archive/.Friends.directory/Emma/cur
Move (365) files to Mail.archive/.Friends.directory/Alvin/cur


It looks for cur, new, and tmp directories but by default only makes new directories as needed which is why tmp and new don't show up in the output. It shells out to the find command for finding the directories and then finding the files within the directories which can make it a tad slow for small directories but I hope it will make it robust for larger problems such as yours.

The initial find (for all the cur, tmp and new) directories can be a bit slow if you want to play around in pretend mode. That's why I added the --dirs and --list options. They allow you to cache the list of directories in a file:
Code:
 $ archive-maildir --list Mail/ > dirs.txt
$ cat dirs.txt | archive-maildir --dirs --pretend --older $ODAYS  Mail/

You can also put in greps after the cat to only deal with a subset of the directories at a wack.

I'll post the code if you are interested.
Back to top
View user's profile Send private message
carpman
Advocate
Advocate


Joined: 20 Jun 2002
Posts: 2202
Location: London - UK

PostPosted: Sat May 26, 2007 2:07 pm    Post subject: Reply with quote

Hello, been trying to get this working but having 2 problems:


Working on multiple user mailboxs uses one command


using suggestion earlier in thread

Code:

archivemail -n -d360 -s '' -o /home/mail-bak ~/.maildir/*
archivemail: '/root/.maildir/.*': no such file or directory



Of course i have no .maildir in root so would need to excluded that


Any suggestion getting this to work?
_________________
Work Station - 64bit
Gigabyte GA X48-DQ6 Core2duo E8400
8GB GSkill DDR2-1066
SATA Areca 1210 Raid
BFG OC2 8800 GTS 640mb
--------------------------------
Notebook
Samsung Q45 7100 4gb
Back to top
View user's profile Send private message
adsmith
Veteran
Veteran


Joined: 26 Sep 2004
Posts: 1386
Location: NC, USA

PostPosted: Sat May 26, 2007 2:22 pm    Post subject: Reply with quote

Are you sure "~/.maildir/*" is correct? What about just "~/.maildir" or "~/.maildir/.*"?

In the Maildir standard, any sub-folders are named with leading dots, so * would not catch them.

E.g., under ~/.maildir you may have
.folder
.folder.subfolder
courierimapkeywords

archivemail only sees that one file with the glob you use, but this is a just a dumb cache file and not a mail folder.
Back to top
View user's profile Send private message
carpman
Advocate
Advocate


Joined: 20 Jun 2002
Posts: 2202
Location: London - UK

PostPosted: Sat May 26, 2007 2:23 pm    Post subject: Reply with quote

Ok realised that i should have /.* so tried again with:

Code:

archivemail -n -d360 -s '' -o /home/mail-bak ~/.maildir/.*


Also tried after create dir /root/.maildir

Code:

archivemail -n -d360 -s '' -o /home/mail-bak ~/.maildir/.*
/root/.maildir/.: I would have archived 0 of 0 message(s) in 0.0 seconds
/root/.maildir/..: I would have archived 0 of 0 message(s) in 0.0 seconds

_________________
Work Station - 64bit
Gigabyte GA X48-DQ6 Core2duo E8400
8GB GSkill DDR2-1066
SATA Areca 1210 Raid
BFG OC2 8800 GTS 640mb
--------------------------------
Notebook
Samsung Q45 7100 4gb
Back to top
View user's profile Send private message
adsmith
Veteran
Veteran


Joined: 26 Sep 2004
Posts: 1386
Location: NC, USA

PostPosted: Sat May 26, 2007 2:35 pm    Post subject: Reply with quote

uhh.. so what's actually there?

what's the output of
"find ~/.maildir -maxdepth 1"
Back to top
View user's profile Send private message
carpman
Advocate
Advocate


Joined: 20 Jun 2002
Posts: 2202
Location: London - UK

PostPosted: Sat May 26, 2007 3:01 pm    Post subject: Reply with quote

adsmith wrote:
uhh.. so what's actually there?

what's the output of
"find ~/.maildir -maxdepth 1"


Out put is:

Code:

find ~/.maildir -maxdepth 1
find: /root/.maildir: No such file or directory


It is just looking in dir of root all the time?


There are 3 users on system root and one user and one for sysadmin, root does not have .maildir and does not need one.


Both sysadmin and user do have .maildir though only user has mail old enough to need archiving at moment. If i run archivemail on the user only it works fine:

Code:

archivemail -n -d465 -s '' -o /home/mail-bak /home/sara/.maildir/.*
/home/sara/.maildir/.: I would have archived 178 of 865 message(s) in 2.7 seconds
/home/sara/.maildir/..: I would have archived 0 of 0 message(s) in 0.0 seconds
/home/sara/.maildir/.Clients: I would have archived 0 of 0 message(s) in 0.0 seconds
/home/sara/.maildir/.Drafts: I would have archived 43 of 143 message(s) in 0.3 seconds
/home/sara/.maildir/.Leads: I would have archived 0 of 0 message(s) in 0.0 seconds
/home/sara/.maildir/.Personal: I would have archived 30 of 205 message(s) in 0.6 seconds
/home/sara/.maildir/.Sales: I would have archived 0 of 0 message(s) in 0.0 seconds
/home/sara/.maildir/.Sent: I would have archived 768 of 2762 message(s) in 4.6 seconds
/home/sara/.maildir/.Pub: I would have archived 110 of 692 message(s) in 2.0 seconds
/home/sara/.maildir/.Support: I would have archived 0 of 0 message(s) in 0.0 seconds
/home/sara/.maildir/.Trash: I would have archived 0 of 1 message(s) in 0.0 seconds
/home/sara/.maildir/.VirusAlerts: I would have archived 0 of 0 message(s) in 0.0 seconds
/home/sara/.maildir/.spam: I would have archived 14 of 68 message(s) in 0.2 seconds


It just won't do this if try running on multiple user .maildir ?


.maildir is there for users:

Code:

 ls -la /home/sara_davis/
total 1275340
drwxr-xr-x  7 sara sara       4096 May 24 02:46 .
drwxr-xr-x  7 root       root             4096 May 26 14:35 ..
-rw-r--r--  1 sara sara        127 May 23 14:46 .bash_logout
-rw-r--r--  1 sara sara        193 May 23 14:46 .bash_profile
-rw-r--r--  1 sara sara        551 May 23 14:46 .bashrc
drwx------ 18 sara sara       4096 May 26 15:12 .maildir

_________________
Work Station - 64bit
Gigabyte GA X48-DQ6 Core2duo E8400
8GB GSkill DDR2-1066
SATA Areca 1210 Raid
BFG OC2 8800 GTS 640mb
--------------------------------
Notebook
Samsung Q45 7100 4gb
Back to top
View user's profile Send private message
adsmith
Veteran
Veteran


Joined: 26 Sep 2004
Posts: 1386
Location: NC, USA

PostPosted: Sun May 27, 2007 12:25 pm    Post subject: Reply with quote

Yes, of course. ~ expands to $HOME of the *current* user. If you want to do this as a cron job, it must be the user's cron job (or you muck aroundwith sudo in your cron script).
Back to top
View user's profile Send private message
carpman
Advocate
Advocate


Joined: 20 Jun 2002
Posts: 2202
Location: London - UK

PostPosted: Sun May 27, 2007 4:28 pm    Post subject: Reply with quote

adsmith wrote:
Yes, of course. ~ expands to $HOME of the *current* user. If you want to do this as a cron job, it must be the user's cron job (or you muck aroundwith sudo in your cron script).



Ok but as i am system admin i need to run this on all users mail and want to do it with one cronjob not individual cron jobs for each user?
_________________
Work Station - 64bit
Gigabyte GA X48-DQ6 Core2duo E8400
8GB GSkill DDR2-1066
SATA Areca 1210 Raid
BFG OC2 8800 GTS 640mb
--------------------------------
Notebook
Samsung Q45 7100 4gb
Back to top
View user's profile Send private message
adsmith
Veteran
Veteran


Joined: 26 Sep 2004
Posts: 1386
Location: NC, USA

PostPosted: Sun May 27, 2007 6:49 pm    Post subject: Reply with quote

OK, this is no longer an archivemail problem, but a combination of bash and cron.

First, you should go google for "Advanced bash scripting guide" for some pointers on scripting.


some cobination of archivemail, chown, and maybe sudo will do what you want, I think.


Although... I'm not sure why you want root to cycle other user's mailboxes, since users generally want complete control of their own mailboxes. The users could always run their own cron job...
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Networking & Security All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum