abcde hogging hard drive - but where?

eduardo451

I just tried ripping a CD w/ abcde, for the first time in awhile. Before doing so I checked my disk space w/ df, which showed just under 1 GB of space left (86% of 6GB shown as used). When ripping the 3rd track w/ cdparanoia, abcde shut down, w/ a message that there was no more space left. Sure enough, when I ran df again, it now said that the Use% had jumped to 99. When I checked my home directory, the temporary abcde directory and the album directory containing the first 2 mp3s amounted to only about 20 MB (the mp3 of track 1 contained only the first minute, whereas track 2 was in its entirety, but that's antoher issue). My home directory as a whole has not expanded. I have checked cache and tmp directories (that I know about) and have found nothing. Where is the crap that's taking up nearly a GB of my space? Any ideas?

AmosMutke · Posted: Tue Dec 13, 2005 2:01 am Post subject:

use "du" and just limit the max-depth. It will still give you the total for directories, but only give totals at a specified depth.

example

eduardo451 · Posted: Tue Dec 13, 2005 7:02 am Post subject:

Thanks. I've discovered that my /var/log directory is over 2 GB. A closer look reveals:

AmosMutke · Posted: Tue Dec 13, 2005 9:34 am Post subject:

You have got many more (esp large) files in your log directory. I

/var/log/messages contains all the output from your system logger. Technically it's not needed, but it's good to have if you need to look back for some reason... like if you suspect someone tried to break into your system. I archive mine once a year.

I don't know what the kernel.log is exactly. is there a kernel logging feature in the kernel? (grep -i log /usr/src/linux/.config) didn't list anything suspicious... If there is, then you can prob disable this. You don't need it unless you're trying to do some debugging.

I don't know what's creating syslog either. (I dont' have it) what does

eduardo451 · Posted: Tue Dec 13, 2005 1:39 pm Post subject:

Doing

AmosMutke · Posted: Tue Dec 13, 2005 2:56 pm Post subject:

eduardo451 · Posted: Tue Dec 13, 2005 11:24 pm Post subject:

eduardo451 · Posted: Wed Dec 14, 2005 1:09 am Post subject:

I've found this thread here

[url]
https://forums.gentoo.org/viewtopic-t-357192-highlight-kern.html regarding kern.log
[/url]

From previous info I posted, I ran

eduardo451 · Posted: Wed Dec 14, 2005 2:02 am Post subject:

I deleted my kern.log file, but when I run df there is no difference in available disk space.

eduardo451 · Posted: Wed Dec 14, 2005 5:43 am Post subject:

Ok, it's official. Abcde totally bloats my log files. When I compressed my /var/log/messages, I stopped sysklogd, and did not start it back up before ripping a cd. Ran abcde, no problem, plenty of room to spare. I start sysklogd back up, rip another cd, and half-way through I'm told I've run out of space. I discover that in /var/log, my messages, kern.log, and syslog files are 717MB each, and when I tail them I get the following:

yabbadabbadont · Posted: Wed Dec 14, 2005 5:58 am Post subject:

abcde is just a bash shell script. It must be cdparanoia that is filling up your logs. It looks suspiciously like debugging output to me. Did/do you have debug in your use flags and nostrip in your features when you emerged it?

eduardo451 · Posted: Wed Dec 14, 2005 6:35 am Post subject:

I have never had debug in my use flags, and this is the first I've heard of nostrip. I'm pretty sure cdparanoia was just a standard install. Also, in my abcde.conf file, there is nothing listed for CDPARANOIAOPTS. Do you think it would help to try reemerging cdparanoia with -debug at the command line? I imagine I don't want to put -debug in my use flags.

AmosMutke · Posted: Wed Dec 14, 2005 9:22 am Post subject:

if you use the -v option, it will tell you what use flages you used to install.

red flags are on, blue are off, if any flag is followed by a *, then it is different from when you installed,

i.e.

eduardo451 · Posted: Wed Dec 14, 2005 4:41 pm Post subject:

eduardo451 · Posted: Thu Dec 15, 2005 1:27 am Post subject:

I've reemerged cdparanoia w/ -debug, but there's no change. It still fills up my log. But the good news is, it is now log instead of logs. I looked at my syslog.conf file for the fist time, and apparently kern.log and syslog are enabled by default. So I commented those lines, and now there's just /var/log/messages.

AmosMutke · Posted: Thu Dec 15, 2005 7:40 am Post subject:

So at this point you have tackeled the duplicate logs problem and now it's a question of getting rid of all the extra output that seems to be dumped in /var/log/messages when you burn a cd.

Am I right?

If yes, then I can't help you. I don't use cdparanoia. Maybe list your cd drive info with

eduardo451 · Posted: Thu Dec 15, 2005 2:07 pm Post subject:

You are correct. Now it's just a matter of getting rid of the extra output.

Here's the info:

rpmohn · Posted: Tue Dec 20, 2005 1:25 pm Post subject: Same problem with Sound-Juicer

Hi, I just started having this problem myself while ripping a bunch of CDs using Sound-Juicer. I recently upgraded my kernel from 2.6.13-gentoo-r4 to 2.6.14-gentoo-r4 and I'm seriously wondering if this is something to do with the 2.6.14 sources?

I've been ripping CDs for years and haven't noticed anything like this. Now after just a few CDs the kern.log file gets unmanageably huge!

I wonder if I've recently upgraded syslog-ng as well. That is another possibility I will look into now...

-RPM

eduardo451 · Posted: Tue Dec 20, 2005 10:42 pm Post subject:

Hi rpmohn. Welcome to the club!

I can tell you it is not syslog-ng, as I do not have it on my system. But I too have upgraded to 2.6.14, so that very well may be the issue. Have you tried going back to a different kernel yet?

AmosMutke · Posted: Wed Dec 21, 2005 2:07 am Post subject:

I'm using gentoo-sources 2.6.14-r2 with no such trouble.

what ever is filling your /var/log/messages is probably repeating the same thing over and over.... can you cut a small sample and paste it here... Let us look at it and see if we can catch the criminal...

eduardo451 · Posted: Wed Dec 21, 2005 4:48 am Post subject:

I've switched back to 2.6.13-r5 and the problem is gone. No more bloated log file. I have no idea why, but it worked for me.

AmosMutke · Posted: Wed Dec 21, 2005 4:59 am Post subject:

I'm not a shell scriping guru, but I would ask the kind people who read this forum for a script that will give you a list of differences in the files... maybe you selected different options in the two kernel configs.

The "diff" command is probably what your looking for.

rpmohn · Posted: Wed Dec 21, 2005 3:05 pm Post subject:

Great idea to try switching back to a 2.6.13 kernel. Sadly, I had removed my last 2.6.13 kernel just before reading your suggestion

.

On a positive note, however, I've narrowed the problem down even further

! I ripped a new CD last night to try to get some examples of the error messsages, but when I checked the kern.log file, there were no errors! Now I was really confused, but then I remembered that the CDs I was ripping over the weekend were rather old and had been used quite a bit by my young son. There were even a few with patches that couldn't be read at all. So I put in an old Leadbelly CD to rip, and lo and behold, there were the errors choking up my kern.log file!!

So, the problem relates to bad reads from a CD device. I can't test if it has anything to do with the kernel version or not. Here are some choice lines from the /var/log/kern.log file:

eduardo451 · Posted: Thu Dec 22, 2005 4:37 am Post subject:

If it's really related to bad reads, then all of my CDs are messed up! But for me it didn't seem to matter what kind of shape my CDs were in, I still got output like you did. Are we sure it is truly "error" messages, and not just reporting every little thing happening?

As far as any differences between my kernels, I ran diff and found no true differences. There are some options that are simply not available in the other kernel, but that would not explain why we are experiencing this when others are not.