Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Implementing an inteligent SAN
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Networking & Security
View previous topic :: View next topic  
Author Message
ninjabadger
Apprentice
Apprentice


Joined: 23 Apr 2003
Posts: 270
Location: Scotland

PostPosted: Mon Feb 06, 2006 4:32 pm    Post subject: Implementing an inteligent SAN Reply with quote

I am looking to implement a SAN and have been looking and network and cluster filesystems in order to do it.

I am getting a bit stuck as to which software would be best (if any) for the problem. So here is the problem if anybody has any suggestions (This is taken from a lustre mailing list post i made so forget about the lustre specificness, the problem is the same and i am looking for any open source software solution)

I am looking for a distributed filesystem to manage data accross four sites with varying connection speeds (up to 2MBit Fibre and as low as 2Meg DSL for smaller site). THe setup i would like is as follows:

4 identical linux machines with equal sized hard disks in mirrored raid. All machines would be data stores and clients (if I understand a client correctly to be a kind of access point for the data). There would obviously be a Metadata server running on one of these and perhaps a failover. These machines would be networked by VPN.

I would like data replicated to all sites and for data to propogate through the system. How I am thinking this will work is as follows:

A user at any of the sites accesses the filesystem. It locks any files on the Meta server and performs the work to be done, reading and writing from the closest storage target (on the same machine as its client).
When changes are applied the Metadata keeps track that the latest version of the file is on that particular storage target and when it is requested it is replicated to whoever requested it. Full replication of any oputstanding files can then be done overnight so that network bandwidth is kept down dusring the day.

So basically i want to replicate on the fly only when required, then run a cron job or some other timed function to do the rest later. Hopefully you understand what I am trying to do.

Is this possible?

Is this possible with Lustre?
_________________
Is Microsoft Stronger?

No, no, no. Quicker, easier, more seductive.
Back to top
View user's profile Send private message
madchaz
l33t
l33t


Joined: 01 Jul 2003
Posts: 995
Location: Quebec, Canada

PostPosted: Mon Feb 06, 2006 4:53 pm    Post subject: Reply with quote

You might want to look into openmosix's distributed file system. I am not 100% sure it will fit what you're trying to do, but it just might.
_________________
Someone asked me once if I suffered from mental illness. I told him I enjoyed every second of it.
Back to top
View user's profile Send private message
ninjabadger
Apprentice
Apprentice


Joined: 23 Apr 2003
Posts: 270
Location: Scotland

PostPosted: Tue Feb 07, 2006 11:25 am    Post subject: Reply with quote

Openmosix seems like a clustering app for processing and i cant find much information on the file side of things.

Do you have any links?
_________________
Is Microsoft Stronger?

No, no, no. Quicker, easier, more seductive.
Back to top
View user's profile Send private message
m3_del
n00b
n00b


Joined: 14 Jul 2005
Posts: 37

PostPosted: Fri Jun 09, 2006 9:43 pm    Post subject: Reply with quote

I am also looking for a good replication solution for a similar set up

We will be running a 2 server iSCSI SAN's setup. We want to be able to replicate with HD Blocks, not file based to save on the amount of data being transfered over (someone adds a word to a txt doc, it would transfer the changed block, not the whole file). Anyone know of something that would be effective for this sort of setup?
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Networking & Security All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum