Network services down or unable to restart peridically??

carlos123

The last few days I have noticed that network related services seem to stop on their own for some strange reason. I can usually restart them once I notice that they are down but would appreciate any insight for why this might be happening from anyone.

Take my net.eth0 service. These days it's operating just fine but once in a while when I am away from my computer I come back to it and find it down for no apparent reason. So I have to restart with ....

/etc/init.d/net.eth0 stop
/etc/init.d/net.eth0 start

Then there is my ntpd daemon. It's even weirder with that one. Sometimes it goes down even when my net.eth0 service is up. And talk about weirdness. Check out the following command line attempts to get it working again...

avenj · Posted: Tue Mar 18, 2003 5:37 am Post subject:

Your second problem is because our init system keeps track of started services. /etc/init.d/<servicename> zap

Crg · Guru Joined: 29 May 2002 Posts: 345 Location: London

avenj is right for part 2. For part 1, what does dmesg tell you? What network card do you have? I'm asuuming from the questions you won't have changed mtu etc...

carlos123 · Posted: Tue Mar 18, 2003 5:48 am Post subject:

Thanks avenji. Just before I read your post, in an act of desperation, I finally got ntpd to start again by doing the zap thing.

Can you or some explain the sentence "Your second problem is because our init system keeps track of started services." as it relates to trying to stop a service and having Gentoo report back that it's already started? Even though it clearly isn't?

Does that mean that if a service is stopped abruptly by an unknown failure that init will still see it as started unless I zap it?

Does this mean that unless a service is stopped nicely and cleanly through the usual "/etc/init.d/<service> stop" routine that init will be thrown off kilter and may return weird messages such as a service already being started when there is no such service actually running?

Am I understanding that correctly? Can you or anyone else expand on what Gentoo does underneath with all this a bit more?

Whatever I learn I will put into the ntpd Documentation Tips and Tricks that I posted on ntpd.

As for these services stopping all of a sudden for no reason I guess I will just wait and see if I can catch them in the act. Although that might be difficult. They have NEVER stopped on me when I am in front of the computer. Only when I am away from it for an extended period of time - say at least an hour.

Thanks again.

Carlos
_________________
I'm not yet a real Guru so if you are a newbie here and want to teach me a thing or two, please do! I'm still learning just like you.

avenj · Posted: Tue Mar 18, 2003 5:54 am Post subject:

carlos123 · Posted: Tue Mar 18, 2003 5:56 am Post subject:

I hope it's okay to post such a long message but here is the output of dmesg. If anybody sees anything weird I would appreciate hearing about it.

I have always thought it weird that Gentoo reports "ds: no socket drivers loaded!" but have never been able to see my way around that particular warning. I bolded that string in the following output for easy reference.

carlos123 · Posted: Tue Mar 18, 2003 6:11 am Post subject:

I just thought of something that might contribute to this weird stopping of services when I am away from the computer.

My on-system clock battery might be dead or dying on me. I think this might be the case because before I started using ntpd my clock was WAY off. And very erratic. I just never took the time to get a new battery and replace mine (bit of a hassle).

Now that I have been using ntpd it usually sets my clock just fine and keeps it accurate so now I have even more reason to ignore my clock battery state.

Could a dead clock battery somehow trigger a service stopper when ntpd does an update of some kind which in turn might also cause my net.eth0 service to stop as well? I have noticed that sometimes ntpd reports that because my clock was off by so much (after such a stoppage) that it refuses to start unless I bring the clock time up to a more or less accurate setting first. Before continuing my efforts to start it.

Could a clock glitch that happens in connection with my screen saver cause a shut down of ntpd and cascade into shutting down my network service too?

I know I am just fishing in the dark here but I thought I would pass that theory by you all in case there might be some water there.

Thanks.

Carlos

PS. Is there any pressing reason for me to change my clock battery

.
_________________
I'm not yet a real Guru so if you are a newbie here and want to teach me a thing or two, please do! I'm still learning just like you.

carlos123 · Posted: Tue Mar 18, 2003 9:56 am Post subject:

I just came back from an absence from my computer of about 2 or 3 hours and although the net.eth0 service was up and okay the ntpd service was down again.

Here are the last lines in the ntpd log file located at /var/log/ntpd.log...

carlos123 · Posted: Tue Mar 18, 2003 10:17 am Post subject:

Here is my new attempt to get ntpd running again. My net.eth0 is still running okay throughout this attempt.

carlos123 · Posted: Tue Mar 18, 2003 10:25 am Post subject:

Well, restarting the computer didn't do any good this time. Here is my latest attempt to restart ntpd after the reboot. My network was up and running just fine.

bLanark · Posted: Tue Mar 18, 2003 10:42 am Post subject: Something else

I think that ntpd stops when the local computer time goes too far from the real time. It can't/won't change the time except by a small amount.

So, possibly the real-time clock is very badly broken, or maybe something else is changing the time - are there any other applications running? What does "ps -ef" (or "ps aux") say? Do any of them change the time?

When ntpd fails to start, some servers don't let you connect without authorisation, so it could be that. Or maybe your dns is shot and you can't resolve the names, or maybe your eth0 is broken and so the requests or responses get lost.

With eth0 going down, next time it happens capture the output from dmesg and post it here.

Are you sure you've got the correct kernel module for your eth0? What does it say about your card in /proc/pci ("less /proc/pci")? What does lsmod say, or is support compiled into the kernel?

Sorry, lots of questions, a few ideas but no answers!

bLanark
_________________
.sig: access denied

carlos123 · Posted: Tue Mar 18, 2003 12:12 pm Post subject:

Thanks for your input bLanark. Good questions that you asked.

I don't want to get too heavy into this in the sense of going way deeper than I need to go so I will have to limit myself to working out the answers to those questions that seem to have the best chance of being relevant to the problem.

I have been looking through bunches of NTP documentation on the Internet and it's like going through the U.S. Tax Code. Ridiculously complex!

Just a few comments on your questions. I know the DNS is working because I can ping to the domain names of the servers I am using and name resolution works.

I know too that, although my eth0 service goes down too sometimes, it has been fine during my attempts to get ntpd started again. I can surf just fine and otherwise use the Internet while not being able to start up ntpd.

The output from dmesg is listed on a previous message I posted on this thread.

As for other applicatioins running there are lots of them. I am running inside of KDE and I have KMail, Konqueror, and several Konsoles open while trying to get ntpd started again. Your question does give me an idea though. I will remove kdm as my start up manager and go back to just the command line login and try starting ntpd that way if I can't work it out while using KDE.

I will also have to try and bootup with Redhat and see what options and things are set up there as a reference point. Assuming ntpd is working okay there, if Redhat uses it too. Though it's going to be a bit of a pain to translate the way Redhat does it and apply it to Gentoo's custom init scripting wrapper type of service starting and stopping.

Support for my NIC is not built into the kernel. I load the 3c59x module into the kernel at bootup by placing "3c59x" on a seperate line in the modules.autoload file. If it was not loaded properly I would not be able to surf the Internet.

Your questions have given me some things to track down and try bLanark. I appreciate that.

If you want to know anything else or still believe it would be helpful to see the output from lsmod for example, just let me know and I will post it.

Carlos
_________________
I'm not yet a real Guru so if you are a newbie here and want to teach me a thing or two, please do! I'm still learning just like you.

carlos123 · Posted: Tue Mar 18, 2003 9:58 pm Post subject:

NOTE: I am posting ongoing follow-ups so that those reading this thread and who are new to Gentoo might learn from what I am experiencing. As well as allowing passerby's more opportunity to comment on anything they might see.

Last night, with ntpd down and not running and indeed unable to be run, I logged out of KDE and left the computer with the KDM login manager on the screen.

This morning I logged in as carlos and everything is working just fine. Internet connectivity and ntpd are BOTH up and running!

When looking at the active processes there are three ntpd processes running!! All owned by root.

I am totally and completely without a clue. As to why I could not for the life of me get ntpd started as a su'ed root yesterday and despite a reboot and why today when logging back in, it started on it's own. All I did was log in.

From 4:00 AM last night when I went to bed to now the /var/log/ntpd.log file shows absolutely nothing but ...

bLanark · Posted: Tue Mar 18, 2003 11:11 pm Post subject:

carlos123 · Posted: Wed Mar 19, 2003 1:39 am Post subject:

Thanks for the additional tips bLanark. Good stuff! I am keeping an eye on it and will be better prepared to see what is happening underneath now that I have a good grasp of what logs to look into, how to check the battery, and so forth. I am assuming that the system log is the /var/log/everything/* logs. If that is not correct I would appreciate a correction of that assumption.

Anyway one way or another I'll post either a fix or a continuing problem once I see something odd again.

As for being rooted - I assume that means some kind of hacking type penetration of my system. It's possible I guess but highly unlikely since I am running a hardware firewall (Smoothwall) that purports to have never had a single instance of penetration between my Linux box and the Internet. My Linux box IP is not even reachable from off the Internet.

Carlos
_________________
I'm not yet a real Guru so if you are a newbie here and want to teach me a thing or two, please do! I'm still learning just like you.

carlos123 · Posted: Wed Mar 19, 2003 2:39 am Post subject:

Well I had to go and mess with ntpd again! Now I can't get it restarted

. Just like before. Here is my attempt at documenting my actions. Bear in mind that I started with 3 ntpd related processes running as I had mentioned previously.

carlos123 · Posted: Wed Mar 19, 2003 3:00 am Post subject:

I was re-reading through some of the posts in this thread and bLanark something you said about port 123 rung a bell. Here are some log entries from last night from the /var/log/everything/current log file....

carlos123 · Posted: Wed Mar 19, 2003 9:53 am Post subject:

Well wouldn't you know it? ntpd works just fine under Redhat!

Here's are some actions on ntpd running under Redhat:

carlos123 · Posted: Wed Mar 19, 2003 9:32 pm Post subject:

This will be my last post on this thread since I seem to be only talking to myself. The thread seems to have sunk into obscurity.

Last night I did not log out of KDE and left the computer running just fine. With both net.eth0 and ntpd services running.

This morning I come back and both are out again!! No apparent reason. Nothing! Fortunately I was able to zap the ntpd service and restart it again. Here is a recording of my attempt.

bLanark · Posted: Wed Mar 19, 2003 11:31 pm Post subject:

workaround: Why not just set up an hourly cron job to set the date via ntpdate, and forget about the daemon?

Why do you need ntpd anyway? Is your clock inaccurate?

I still suspect that something else is changing the time on your machine as the "kernel time discipline status change 41" error suggests that the time is too far out to change. If this always happens at a particular time, it is probably a running daemon or a cron job.

What happens (to ntpd) if you log out of the box entirely (no display manager either) for a night? What happens if you leave a normal user or root logged in on a console?

(That'll take you 3 days !)

BTW: the "time correction of 7191 seconds exceeds sanity limit" is approx 2 hours. That's a bug drift .... is your timezone set correctly everywhere?

bLanark
_________________
.sig: access denied

carlos123 · Posted: Thu Mar 20, 2003 3:17 am Post subject:

Well I guess it's not my last post on this thread after all

.

Thanks again bLanark!! You've given me some more good input to work my way through.

One reason that I want to use ntpd is that I HATE leaving loose ends and having to tell myself that I cannot otherwise use a piece of perfectly good software on Gentoo when I can use it on other systems like Redhat. To me that spells defeat for Gentoo in my mind and undoubtedly in the minds of others that I recommend Gentoo to.

I mean how would it sound for me to say something like "Yeah Gentoo is a great system and much better than Redhat but...let's see...you can't run ntpd like you can under Redhat"

So my customer asks me why not to which I fiddle and faddle and look at my shoes and say "Well...I really don't know. You just can't". Not only is that an embarrassment to Gentoo but to me as being too incompetent to figure it out. At least in the mind of my customers. Regardless of the reality.

I know that it might seem a bit picky but in encouraging others to try out Linux (and Gentoo) every little such pickiness can make or break the willingness of others to try it. And to pay me to install it on their systems. Or to support them in it's use.

I want to shield the average customer from all the gibberish and all the problems and all the configuration headaches that I have experienced since coming to Linux. Frankly for all the good press I heard about Linux I still can't believe all the hassles associated with it's use (it took me something like two days just to get my USB mouse to work under Gentoo!). My customers just want to be productive and to receive a working system. Not to become Linux hackers. And I so much want to be a part of getting people away from Microsoft and to experience freedom through Linux.

Not because I hate Microsoft but because Linux is trully a great OS once you work your way through all the gibberish, configuration, and software installation hassles that hold it back from mainstream acceptance. At least on the Desktop.

Another reason bLanark is that I don't want to concern myself with how accurate the clock is being. I have been reading a massive thread on this forum about clocks loosing or gaining time under Gentoo (95 posts last I counted). Using ntpd will prevent this for me. Where I won't even have to pay attention to the clock and be left wondering if the clock is accurate today as oppossed to yesterday type of thing. If ntpd is running then it will be accurate.

I want a dependable clock. I *need* a dependable clock. So do my customers. I can't afford to have a Gentoo system clock to be accurate one day and loose or gain time the next.

The thread in question is a real eye opener for me in terms of not depending on the clock alone.

In terms of just sticking ntpdate in a cron job...ntpdate is a deprecated utility which the ntpd web site recommends one's not using anymore. The functionality of ntpdate has been moved entirely into ntpd. It may work...for now. But it's deprecated.

And from my CSS, HTML, and Perl background if something is deprecated I no longer want to use it. If I can help it. Just a pet peeve of mine I guess but then again that's why these projects label things as deprecated. So that users will no longer use the deprecated item.

Which brings me right back to why ntpd won't run consistently and accurately only under Gentoo.....I just don't know yet but I'm not going to give up on this.

Aside from what you have suggested I will be adding my own output comments to Gentoo source code to better tell where the attempt to run ntpd is at (when it refuses to load). I guess you could say that I am going to do a little hacking and debugging of the source code

.

Anyway I'll post my results back to this thread I guess when I have implemented some of your suggestions and tried some other things.

Thanks again bLanark for helping me out with your input. Very appreciated. I hope I didn't come accross as too frustrated. It wasn't personally directed at you.

It's just that lately Linux and more specifically Gentoo has been such an incredible pain in the butt to deal with! It's like something always needs some tweaking or configuration or patching.

I'm just looking forward to the day when I have a solid and working Gentoo system that doesn't need so much of this. I am truly hoping that there is such a day ahead and that I won't come to find out that using Linux is a continual wading through gibberish and just more hacking and tinkering and attempts to keep the whole thing running smoothly.

I keep talking Linux up as being so much better than Windows but a growing and gnawing doubt is beginning to surface within me that maybe it isn't. Windows, for all it's flaws, is so much more imminently useful right out of the box in comparision to Linux (or at least Gentoo) that sometimes I wonder if I made the right choice switching to Linux.

For me, I've spent too much time in Linux to throw it all away and go back. But for my customers? I'm not so sure that I will recommend Linux as much as I would have before I switched myself and came to know it hands on. It's a great OS for certain uses but overall it's one incredible pain to work with!

Thanks for "listening". I needed to get that off my chest.

Carlos
_________________
I'm not yet a real Guru so if you are a newbie here and want to teach me a thing or two, please do! I'm still learning just like you.

carlos123 · Posted: Wed Mar 26, 2003 11:37 pm Post subject:

If you are still "listening" in to this thread bLanark, just wanted to let you know that I tried variations of both things that you suggested. The problem was still there when I changed the display manager to xdm instead of kdm and logged in to a console login prompt as root (although I did not try just keeping the computer logged out).

A BIOS check of the time was fine overnight so it wasn't the battery.

The odd thing is that the problem has apparently disappeared. Entirely!! My net connection and my ntpd service stays on always now. Like it should. And I have not changed the time servers queried at all.

It's as if the problem was never there.

I am just going to enjoy the working system for now and not bother to fiddle with this problem again unless it pops up again. I did emerge some other programs and do other things on my system and it is possible that something that was screwy got fixed in the process. But as to what that is? Who knows.

I suspect that I will eventually face this problem again. If not on my own computer then on that of my customers computers that I install Gentoo on. But I guess I will cross that bridge again when I come to it

Thanks again for your input and help bLanark. Much appreciated.

Carlos

bLanark · Posted: Thu Mar 27, 2003 7:30 am Post subject:

I am still listening. Thanks for the update - very strange, but if it ain't broken, don't fix it!

I'm a little wary about installing gentoo at customer sites - I'd rather rely on redhat (apart from the limited lifetime of each version) or similar, and have something that is tested a little more than what's in portage. I've disabled my machine on countless occasions just by "emerge rsync; emerge -u --deep world".

For a *big* shop, however, I can see gentoo being good, where there are dozens of identical machines and they have the resources to test everything thoroughly before putting it on the desktops (or whatever). It'd be nice to have a customer that big, however.

bLanark
_________________
.sig: access denied

carlos123 · Posted: Thu Mar 27, 2003 1:34 pm Post subject:

Yeah I am still leery myself about putting Gentoo on customers machines myself. Precisely because of these types of quirks that I have experienced.

If things work out I will shortly be giving free seminars on why one might want to go with Linux (in a joint venture with my computer supplier) but have not yet decided exactly how I am going to supply those who want Linux with a working distro. Aside from charging them for an extra hard disk, a whole system, or for the labor to install it of course

.

Redhat is SO.o.o.o.o much easier to install and initially configure than Gentoo. And it has fantastic hardware detection. Yet Gentoo has portage which makes installation of programs and the latest versions SO.o.o.o.o much easier than on Redhat (albeit still a bit problematic with it's own quirks).

I wish there was a Redhatted type of Gentoo with portage

. I may just compile programs under Gentoo and copy them over or make them available to those of my customers who want the binaries. Assuming that is even possible. While installing Redhat on their computers. I am not sure yet.

I might follow the lead of Lindows Linux here and offer to provide binaries to customers for a small fee per month which will include my doing the edit the configuration file thingy over VNC or some such program.

My goal will be to capitalize on the growing curiosity about Linux by offering my services to overcome the negatives associated with Linux. Mainly the installation of it and subsequent package installation and configuration.

The quirks of Gentoo of which there are several are definitely frustrating and very time consuming to deal with. Redhat just works! And works reasonably well. Right out of the box. Without quirks (that I have experienced at least).

Carlos