'Storage' to Replace Traditional Filesystems

Nermal · Apprentice Joined: 18 Apr 2002 Posts: 259 Location: UK

I think it will be impressive when it goes wrong

Xiol · Apprentice Joined: 24 Jul 2003 Posts: 209 Location: UK

I thought that Storage, and Longhorn's FS, were just laid on top of the traditional filesystem.

The database just stores the metadata...

Wait, I have no idea what I'm talking about.

Think it's time to RTFA.

robdavies · Tux's lil' helper Joined: 06 Sep 2003 Posts: 90

Storage looks like an interesting project, wonder how useful it will turn out in practice. Many users don't get on to well with locate, so they may like it.

Way back, UK mainframe firm had a contents addressable file system, might be same sort of idea.

As for M$ and Longhorn, I really do not care what they are planning.

hook · Posted: Tue Sep 09, 2003 12:49 pm Post subject:

yup, seems identical to M$ longhorn :]
but seriously, i think this "fs" could work only for complete noobies (on any platform!!) and people who constantly keep forgeting where they have stuff.
i don't like it, but i'm sure some guys like it ...because if there's absolutely noone who would use it, it wouldn't be there IMO.

also IMO you have to think about this "fs" not just from the "how i use it" side, but also from the "how i make it work" side ...this "fs" will require the user to rename every file he gets into a short description ...this could be VERY awkward ...imagine having millions of mp3 that are listed as numbers and have emtly id2/3 tags ...that's fun!!! ...imagine then the same situation, only not all files are the same filetype ...and now for the final stroke: imagine how you'd describe portage and the /etc/ and /var/ and /dev/ dirs?!? ...imagine litterary trying to "emerge my favourite office suite, you know the one with the little star ...i think it's abbrevated OOo ...no idea why" ...well, i'm not a person who'd enjoy such a "fs" ...but i'm not conviencing anyone of not trying ...i'm just saying i wont
_________________
tea+free software+law=hook

(deep inside i'm still a tux's little helper)

lurid · Guru Joined: 12 Mar 2003 Posts: 595 Location: Florida

krusty_ar · Posted: Tue Sep 09, 2003 4:06 pm Post subject:

I've just ovelooked the idea behing Storage, It could be usefull, but if it were my decision, I would put the effort in this that it's truly new and revolutionary, besides being one of the first mayor OSS proyects that tryes to create something new and better instead of just following M$'s bad example.
_________________
I am Beta, don't expect correct behaviour from me.
Take part of the adopt an unaswered post initiative

Unne · l33t Joined: 21 Jul 2003 Posts: 616

I really don't like the idea at all. It might be beneficial for users like my parents who have a hard time grasping the concept of directories and subdirectories and files in directories, etc. But for anyone who's used a computer for any given length of time I think it'd be a huge hassle.

"Searching" for files is one of the biggest pains in the world, to me. Probably why we have so many tools for it (find, locate, etc.), and it's still often a hassle. The fastest way to locate a file is to put it somewhere and remember where it is, or find where it is and remember that location. I don't often remember filenames exactly, but I do remember the locations of files in my directory tree. I don't need the computer to manage the locations for me, and I'm pretty sure the computer would do a poorer job of it than I can do myself. If I was forced to "search" for every file I used, I'd go insane.

hook · Posted: Wed Sep 10, 2003 9:28 am Post subject:

actually Unne gave me an idea how this would be useful!!!

imagine having a completely normal linux system, which hosts a few complete idiots. you could make a /home2 (or /home_idiots whatever you like) and link a storage pratition to it ...this way the admin and all the normal users would have the normal fs, and the idiot users would use the storage on /home2 (or /home_idiots)

_________________
tea+free software+law=hook

(deep inside i'm still a tux's little helper)

() · l33t Joined: 25 Nov 2002 Posts: 610

[irony]While you're at it maybe you could provide a separate boot with automated installation of packages, for those morons who can't compile programs themselves.[/irony]

There's noone saying that Storage won't provide a way of structuring files similar to hierarchical filesystems, at least if one is to believe the HCI paper: 'Planned augmentation with a category refinement system may produce a hybrid that addresses both object reference and exploration'.

Cossins · Posted: Wed Sep 10, 2003 4:42 pm Post subject:

The idea of Storage (or the Longhorn FS, for that matter) is not to have a whole new kind of filesystem - it is to put layer in between the user and the filesystem which takes care of file destinations, names, permissions, and stuff like that so the user won't have to worry.

I imagine having a button called "Music", clicking on which would bring up a window listing all music files on the harddrive, independant of file types and such. It could then be sorted in subfolders by title, author, release date, etc.
Maybe we will need a new term for folder, maybe a whole new interface for navigating them (making it easier, mind you).

I see a lot of potential in this idea, I hope it develops into something useful...

- Simon

RedBeard0531 · Guru Joined: 21 Sep 2002 Posts: 415 Location: maryland

doesnt reiser4 do something like this. there ia a layer of abstarction between the users and the HARDCORE filesystem structure. It uses a plugin system to allow you to view your files how ever you want.

[sarcasm]
I think that people should access files by using the CHS/LBA values for them. I want coplete control over where my files go. anything else is for noobs
[/sarcasm]

No offence, but i think that several of you are judging something before trying it, or even becomeing well informed. just b/c MS is doing something doesnt make it bad. Gates above all is a business man, I doubt he would waste millions on the r&d to develope a new fs if there was no advantages.

the following is purly conjucture, please feel free to skip it and go to the next post

My gess is that storage wouldnt effect most dirs, such as /dev or /bin. It might provide a frontend (while leaving the backend available) to dirs like /home and /etc. Imagine typing /config/sys/fluxbox/menu to access the systemwide flux menu. Mabey to would start with a differant char than / to signify the diffeances. You could even do something like /media/music/linkin\ park/meteora/Faint.

mabey a /media/seach dir that you could "mkdir faint" to create a dir containing all the media with the word faint. This sorta exists now. take a look at lufs ant the gnetfs. it does this from the gnutella network. it is proof of concept now, tho.

I appologize for my atrcios speling

[/code]
_________________
OH MY GOD! Kenny just killed Kenny!
That Basterd!

ebrostig · Posted: Wed Sep 10, 2003 10:14 pm Post subject:

Nice to see that others are following in Oracle's footsteps

Oracle has had it's iFS or internet File System for quite a few years now. To the regular user it looks like a normal disk or directory under *NIX. All of the contents are stored physically in an Oracle database and the contents can be indexed and searched using any of Oracle Content managment like Spatial, Video, Image or Text.

See Oracle Content Managment

The only culprit is that this is a huge task for a database and requires a lot of resources and requires a database that is not yet available as open source in order to function properly.

Erik
_________________
'Yes, Firefox is indeed greater than women. Can women block pops up for you? No. Can Firefox show you naked women? Yes.'

telex4 · l33t Joined: 21 Sep 2002 Posts: 704 Location: Reading, UK

One thing I don't get; maybe some can answer this...

Storage uses a PostgreSQL database for the filesystem, but also keeps the traditional filesystem heirachy. Does this mean that you do away with reiser, ext etc. and have storage, and then have two "frontends" to it... one that is emulating the traditiona filesystem, and the other that provides a meta-information based system?

ebrostig · Posted: Wed Sep 10, 2003 11:20 pm Post subject:

tdb · Posted: Fri Sep 12, 2003 5:22 am Post subject:

This whole thing reminds me of Evolution's "Virtual Folders" feature.

corrs_fan · Posted: Fri Sep 12, 2003 2:43 pm Post subject:

i just had another thought about this too. What the heck kind of CPU will you need for this

think about it, it has to (re)reference everything all the time, when you create, ammend and delete the "content", the backend must have to monitor this and update itself, which surley means more means more memory usage too. that athlon-64 is looking nicer by the day.

_________________
Some say "The glass is half empty",
I usually say "Eh, There was a Glass.."

paperwings · Posted: Fri Sep 12, 2003 5:29 pm Post subject:

I don't really understand this push for a new type of filesystem anyway. What's wrong with the hierarchical type? I've heard people going one about time-rather-than-filename based systems, etc. They say it is more natural to think of the document you worked on yesterday rather than the document named "x.xxx". Whatever. I say moose muffins. Seems to me that the hierarchical system is more natural. After all, before computers, we filed things in cabinets by name.

What's that sound? It's the sound of these new filesystems flopping when no one uses them... :wq

P.S. it is not necessary to type :wq when finished with post. I will never learn...

ebrostig · Posted: Fri Sep 12, 2003 10:45 pm Post subject:

Tasslehoff · Posted: Fri Sep 12, 2003 11:42 pm Post subject:

What I wonder about is added overhead. How much will there be? I assume there should be some, unless of course the search for the files will be more efficiant thereby decreasing the overhead.

ebrostig · Posted: Sat Sep 13, 2003 1:01 am Post subject:

watersb · Posted: Wed Sep 24, 2003 2:54 pm Post subject:

I am participating in a project to support large data archives of astronomical data.

So far, a simple hierarchy (mapped directly onto a traditional filesystem) serves as a structured storage repository for the raw visibility data, which is several terabytes. We use a simple meta-data index to locate the correct data files and serve them up to users. So far.

In the near future, we will need the ability to model the visibility data as if it were a monolithic data source, from which we may serve sub-images: a virtual sky, with image cut-out service. Jim Gray and Alex Szalay have done this already for optical surveys, and Jim has done this for Earth-observing imagry.

It is a hard problem. Simple and hard. Not-so-simple when you consider the manipulations needed to normalize the data... anyway...

I have experimented with PostgreSQL and Oracle and SQLServer, adding spatial indexes to them; this works OK.

But most science data archives look a lot like filesystems, the users expect this, the enormous amount of legacy data-reduction software expects this.

There are efforts to model the thing exactly like iFS or WebDAV. But I suspect for a specialized data repository like this, we might be able to use something like Reiser4.

My point is that for some applications, data-storage of this nature might be useful. It still isn't clear. Not clear that it isn't a bad idea. Might be a way to simplify.

IBM mainframes have been doing this for a long time...