• Welcome to Overclockers Forums! Join us to reply in threads, receive reduced ads, and to customize your site experience!

Wanted: a compressing filesystem for Linux

Overclockers is supported by our readers. When you click a link to make a purchase, we may earn a commission. Learn More.

klingens

Member
Joined
Apr 3, 2002
Location
Xanadu
I want a compressing filesystem for Linux similar to doublespace or stacker but with some important changes. Preferred format is either gzip or bzip2 but other formats are just as good. It has to compress at least as good as gzip tho.

I don´t want it to use a big-*** file where the stored files are inside, compressed. Instead I want it to compress each normal file itself, I just want them to access transparently. The compressed files are still accessible on the outside, but just in their compressed form. e.g. there is this zlib library. I can use it to access each gzip compressed file. If I want to read samba-manual.html.gz I just call "lynx samba-manual.html"
This library then, transparently, loads and uncompresses the file, since it recognizes the open() call for the filename without .gz: it appends .gz, loads the file, gunzips it and finally creates a temporary uncompressed file so lynx can load it. Lynx doesn´t know anything about the work behind the scenes. For lynx this is just another fopen() call like any other. The trouble with this scheme is, a "ls" or a filemanager still show all the .gz files. and windows programs which access these files via samba are totally confused by all this, since they don´t believe me that a "samba-manual.html" exists since their directory listing doesn´t show one. There´s only a samba-manual.html.gz so they say.

So basically I want the described scheme without the mandatory "gz" endings. does anyone know of such a beast ?
 

moorcito

Member
Joined
Sep 19, 2002
Location
Chicago, IL
Even if you could have the files striped of the .gz extension, Windows programs will still not know what to do with them because the are still compressed. I haven't used samba, but I'm assuming that it's like any file server, and all it will do is serve the files, not transparently decompress and then serve, though one could also assume that you might be able to get it to work like this.

There are a few compressed filesystems for Linux, such as SquashFS, romfs, or zisofs, but they all are based on the zlibc library, which you are familiar with, and they are read-only filesystems. Other than that, I don't know of any other filesystem you might use.

As far as not having the mandatory .gz, from what I've seen gzip, or zlibc will recognizes compressed files by their extension, weither it be .Z, .gz, or . tgz. Try renaming a .gz file to something without the .gz extension and then try to decompress it with gzip/gunzip; though zcat reads the file without any problems.
 

XWRed1

Senior Member
Joined
Oct 31, 2001
Are you sure those are all read-only filesystems?

cramfs (i think thats what they use now) on the PocketPC is read-write, compresses decently I guess.
 

moorcito

Member
Joined
Sep 19, 2002
Location
Chicago, IL
That's what the all the project homepages say. I think it's because they're based off of zlib, Zlibc is a read-only compressed file-system emulation.

Cramfs is the same way, and from what I can tell it's the same thing as romfs (just different names in typical Linux fashion).
 

XWRed1

Senior Member
Joined
Oct 31, 2001
Well in whatever compressed fs they use on the PocketPCs, I can write to the filesystem just fine.
 
OP
K

klingens

Member
Joined
Apr 3, 2002
Location
Xanadu
windows programms will work just fine, since before samba can transfer the files over the network, it will have to open them and read them into memory in linux. at that point, zlibc kicks in an decompresses the file. what the windows programs don´t like is the gz-ending and me requesting a file without it which doesn´t exist

Readonly files are OK, since the stuf is mostly static anyways (lots of ebooks) I want the compressing filesystem to save space on my disk, that´s all

Too bad squashFS compresses things like inodes and directories as well :(