| View previous topic :: View next topic |
| Author |
Message |
eccerr0r
Watchman
Joined: 01 Jul 2004
Posts: 9824
Location: almost Mile High in the USA
|
 Posted: Sun Sep 22, 2024 2:03 pm Post subject: making another computer swap storm... |
 |
|
Well, this was kind of amusing...
I was trying to update my old Atom 1.6GHz (1C2T, 32-bit) with 2GB RAM. It has a 32GB SSD so I decided to swap over NBD to my 24/7 server/shell box with RAID5. I'm using 100Mbit Ethernet to hook up to my network instead of WiFi that I normally do.
I was using -j 12 -l 3 with MAKEOPTS and have distcc working just fine.
However right now it's struggling with sys-devel/llvm -- as llvm-tblgen seems to also want to run -j12, and this of course won't distcc. The rest of the build distcc'd just fine of course.
But, as expected, the poor Atom is thrashing bad:
| Code: |
$ uptime
07:59:50 up 4 days, 23:15, 4 users, load average: 8.14, 8.26, 8.31
$ free
total used free shared buff/cache available
Mem: 2058824 2039076 135904 9600 57132 19748
Swap: 7864304 5224472 2639832
|
Except all of this swap storage is on my server. My Ethernet is madly passing packets back and forth. It's unfortunate as my Ethernet backbone is GbE but the Atom only has 10/100 but it didn't really matter, my server is likewise struggling with disk i/o
| Code: | server$ uptime
07:52:00 up 80 days, 21:27, 2 users, load average: 8.57, 8.97, 9.63
server$ free
total used free shared buff/cache available
Mem: 8026100 3713496 198176 256788 4114428 3751172
Swap: 20971512 1534016 19437496
server$ sudo iotop -o -b -n 1
Total DISK READ: 493.43 K/s | Total DISK WRITE: 0.00 B/s
Current DISK READ: 246.71 K/s | Current DISK WRITE: 0.00 B/s
TID PRIO USER DISK READ DISK WRITE SWAPIN IO COMMAND
18679 be/4 me 123.36 K/s 0.00 B/s 0.00 % 99.99 % nbd-server [pool]
18785 be/4 me 370.07 K/s 0.00 B/s 0.00 % 99.99 % nbd-server [pool]
1094 be/4 root 0.00 B/s 0.00 B/s 0.00 % 99.99 % [md127_raid5]
|
At 99% IO utilization, yeah, my server is being hammered with seek requests and of course throughput drops like a rock on rust spiners. I'm kind of surprised the server, though only with 8GiB RAM, isn't caching this better than it seems to be. But it is caching a good 4GiB or so of it, alas, it still has its other duties.
Incidentally the machines are still somewhat responsive through ssh, though I ioniced the Atom's llvm-tblgen processes. There are occasionally long waits but not way too bad.
What I should have done is run nbd to a ramdisk on my 24GiB or 64GiB machine... perhaps I should kill this and try again... or heck I don't have to kill, just need to set up a higher priority ramdisk nbd swap and swapoff the raid5 nbd swap...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
| Back to top |
|
 |
pingtoo
Veteran
Joined: 10 Sep 2021
Posts: 1252
Location: Richmond Hill, Canada
|
| Posted: Sun Sep 22, 2024 3:38 pm Post subject: |
|
|
May be make NBD ramdisk for /var/tmp (or /var/tmp/portage) will help?
I don't know why your io show nbd-server 99% busy on "read", unless your raid is being rebuild, it does not make sense no able to keep up on reading. page read/write usually are 4K activity so your NBD (client) block size is not matching? or is the NBD server serving a RAW disk (partition) or on file? |
|
| Back to top |
|
|
eccerr0r
Watchman
Joined: 01 Jul 2004
Posts: 9824
Location: almost Mile High in the USA
|
| Posted: Sun Sep 22, 2024 3:47 pm Post subject: |
|
|
disk image for swap. Swap is being read virtually randomly and rust spinners suck at random i/o. I suspect having 512K chunks on the RAID means it needs to read much more than needed isn't helping but I was hoping cache mitigated that somewhat.
Well, I ended up creating a nbd ramdisk swap on my 24GiB machine, and it's making a huge difference...it's actually making forward progress now since there's no more seek penalty on random swap i/o. Of course it still needed to re-read the things that got swapped to the rust spinners, but eventually they'll get pulled out. Load average on the build machine has dropped to around 4 and it's actually running stuff that's not llvm-tblgen now!
Alas still not completely using 10MB/sec yet.. still trying to pull data off the swap on the server RAID and use the RAMdisk swap on the third machine...
Next I wonder if a zramdisk on the 8GiB machine would be as good as of using a regular ramdisk on the third machine...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Gentoo Chat
