Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Scheduler questions.
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
madbiker
Guru
Guru


Joined: 12 Oct 2003
Posts: 439
Location: Victoria, BC, Canada

PostPosted: Mon Feb 07, 2005 10:23 pm    Post subject: Scheduler questions. Reply with quote

I searched the forums, but didn't really see much that really explained things well.

So from what I've read, there are I/O and CPU schedulers, and there are a few major choices for each. Different kernels include different schedulers in them, and different ones work better for different purposes.

Using ck sources, I compiled in support for all 3 schedulers that I was given the option for. I can change which one I'm using by passing elevator=deadline or elevator=cfq or whichever one I want in my grub.conf. So these are I/O schedulers correct? Is there any way to determine which one I'm currently using? How about CPU schedulers? How do I know which one I'm using, how do I change?

Finally, are there generally accepted standards for schedulers for desktop use and server use? I imagine some of them are going to be considered "unstable" or in "testing" phases, but while being unstable also provide better desktop performance (much like ck, love and nitro kernel sources)?

Thanks for any light you can shed on the subject.
Back to top
View user's profile Send private message
choco
n00b
n00b


Joined: 10 Jun 2003
Posts: 49
Location: Jackson, MI

PostPosted: Tue Feb 08, 2005 1:37 am    Post subject: Reply with quote

I'm interested in learning more about the various schedulers too. http://kerneltrap.org has been an informative site for this sort of thing, although it's so technical (it refers directly to the email lists of kernel developers).

I'm not as sure about CPU schedulers, but you can tell which I/O scheduler is in use by looking at the kernel boot log. Doing a
Code:
dmesg | grep sched
should give you that. The general guideline I've heard is that anticipatory I/O or deadline is good for servers and databases because of better throughput, while cfq is good for the desktop because no program is able to slow the system down by monopolizing I/O.

I don't believe the CPU scheduler can be chosen by a boot parameter - it requires a patch. I don't know what CPU schedulers are available/in development. A selling point of the Gentoo kernel prior to 2.6 was that it had the O(1) scheduler. I don't know if any special CPU scheduler patch goes into the main Gentoo kernel now.
_________________
Who would win in a fight between Sandman and Clayface?
Back to top
View user's profile Send private message
commonloon
Tux's lil' helper
Tux's lil' helper


Joined: 13 Apr 2004
Posts: 133
Location: Marin, CA

PostPosted: Tue Feb 08, 2005 5:33 am    Post subject: a little more on io schedulers Reply with quote

You can find some documentation in /usr/src/linux/Documentation/block where linux is the symlink to your kernel version.

2.6 has the as (default - anticipatory), cfq (completely fair), deadline (good for databases) and noop (good for devices such as usb drives).

You can find out your scheduler via dmesg as described above or in /sys/block/sdb/queue/scheduler on _some_ newer (2.6.10?) or patched kernels. On these kernels you may change it w/o rebooting by echo cfq > /sys/block/sdb/queue/scheduler (sdb is the device in question) for example -- auto-magically you'll then notice it will look like:

porky queue # cat scheduler
noop anticipatory deadline [cfq]

You can adjust io scheduler parameters by adjusting things in: /sys/block/sda/queue/iosched where sda is the device your interested in changing. The parameters and files within the iosched dir vary -- see the respective documentation.... seems more options (and more complex) that elvtune for 2.4. You can change via standard echo value > iosched/param

Hope this helps...
_________________
Calling fly fishing a hobbie is like calling brain surgery a job.
Back to top
View user's profile Send private message
madbiker
Guru
Guru


Joined: 12 Oct 2003
Posts: 439
Location: Victoria, BC, Canada

PostPosted: Fri Feb 11, 2005 2:36 am    Post subject: Reply with quote

Can't believe I missed this reply.

Yes that helps a lot! I wasn't getting any mention of which scheduler I was using with dmesg, but I'm using a 2.6.10 and had that scheduler bit in my /sys. Works great.

Now my only questions relate to CPU schedulers. I'm using ck-sources, so does that mean I'm using the 'staircase' scheduler automagically? Is it simply determined by the kernel patchset you're using, and there's no way to change it other than go to a patchset that's patched with a different scheduler?
Back to top
View user's profile Send private message
steve_d555
Guru
Guru


Joined: 07 Nov 2004
Posts: 458
Location: Belmont, Massachusetts

PostPosted: Fri Feb 11, 2005 5:36 am    Post subject: Reply with quote

I have a question, I hear that Deadline/Anticipatory is better for gaming as it allows it to take up 100%. Does anyone have any thoughts? Has anyone done any benchmarks against CFQ or something? I would be really interested to know, because ive been using Nitro sources with CFQ and was wondering what was better.

Thanks,
--Steve
Back to top
View user's profile Send private message
commonloon
Tux's lil' helper
Tux's lil' helper


Joined: 13 Apr 2004
Posts: 133
Location: Marin, CA

PostPosted: Fri Feb 11, 2005 6:23 am    Post subject: Reply with quote

As far as cpu schedulers, I can't help much as I only know enough to be dangerous. In 2.6 there is O(1) "Ultra-scalable" (damn cool name) scheduler that is the main one (/usr/src/Documentation/sched-design.txt). There is also, I think, the SC or staircase and the PB or priority based; I'm pretty sure that you need to have a specific patchset to have either of the alternative cpu schedulers.

Regarding gaming and schedulers -- haven't seen specific benchmarks.

I'm actually in the middle of doing some benchmarks comparing various schedulers and parameters for a database server. I'll post some results when I'm done, but they may not help w/ destop or gaming.
_________________
Calling fly fishing a hobbie is like calling brain surgery a job.


Last edited by commonloon on Fri Feb 11, 2005 6:38 am; edited 2 times in total
Back to top
View user's profile Send private message
steve_d555
Guru
Guru


Joined: 07 Nov 2004
Posts: 458
Location: Belmont, Massachusetts

PostPosted: Fri Feb 11, 2005 6:32 am    Post subject: Reply with quote

OK, Ya I heard somewhere that the Staircase/CFQ is not very good for gaming as it doesnt allow users to take 100% of the CPU or something.
I heard that the best setup would be the default Scheduler with Deadline in gentoo sources. I'm not sure, but I might try testing it with a new rig I'm getting just to try and get the full performance outta my 6600GT :wink:

--Steve
Back to top
View user's profile Send private message
oggialli
Guru
Guru


Joined: 27 May 2004
Posts: 389
Location: Finland, near Tampere

PostPosted: Fri Feb 11, 2005 8:42 pm    Post subject: Reply with quote

Hmm, you probably won't gain anything in games using those things compared to what you gain using cfqv2 and a recent staircase in multitasking/general desktop usage. For example I get 64,1 fps in Doom3 timedemo demo1 usecache second run with 1024x768, 4xAA, 8xAF, Ultra Quality. I think that is enough, and with the stock scheduler my desktop feels a lot more choppy.

Also, batch scheduling absolutely rocks.
_________________
IBM Thinkpad T42P - Gentoo Linux
Back to top
View user's profile Send private message
masseya
Bodhisattva
Bodhisattva


Joined: 17 Apr 2002
Posts: 2602
Location: Baltimore, MD

PostPosted: Fri Feb 11, 2005 9:33 pm    Post subject: Reply with quote

Moving from Gentoo Chat.
_________________
if i never try anything, i never learn anything..
if i never take a risk, i stay where i am..
Back to top
View user's profile Send private message
fallow
Bodhisattva
Bodhisattva


Joined: 08 Jan 2004
Posts: 2208
Location: Poland

PostPosted: Sat Feb 12, 2005 11:22 am    Post subject: Reply with quote

about O(1),Staircase and SPA Family also EBS and Nicksched is in many of threads on this forum , It was been talked many times but casue of many of threads it`s difficult to search every sentence/post ..but possible :D :)

maybe someone will be interested in easy io scheduler changing script :) - I`m using it all the time

https://forums.gentoo.org/viewtopic.php?p=1945476#1945476

and IMHO : https://forums.gentoo.org/viewtopic.php?p=2053237#2053237

also about bonnie++ small comparison on whole reiser4 system beetwen staircase / O(1) / nicksched .
https://forums.gentoo.org/viewtopic.php?p=2076954#2076954
( results : http://vivid.dat.pl/fallow/sched_test/results.html )
cheers :)
_________________
"Time is a companion that goes with us on a journey. It reminds us to cherish each moment, because it will never come again. What we leave behind is not as important as how we have lived" J-L. Picard ;)
Back to top
View user's profile Send private message
commonloon
Tux's lil' helper
Tux's lil' helper


Joined: 13 Apr 2004
Posts: 133
Location: Marin, CA

PostPosted: Sun Feb 13, 2005 2:32 am    Post subject: Some benchmarks comparing schedulers Reply with quote

...Thanks Fallow for the info....

What follows is some of the benchmarks I've been running on a new server. They are only benchmarks (for fun). I don't really think they show much difference between schedulers (I'm sticking w/ deadline, since this is a database server). I'm also testing using a SQL test script to test closer to the real world. The server's specs are:

Tyan 2882 mobo
16GB ram
2x 250 Opterons
2x10K 73GB RAID1 OS
6x15K ~200GB RAID10 data
gentoo-dev-sources 2.6.9-r14

I have booted w/ mem=4G to make the tests run a bit quicker, as w/ 16GB of ram running a non-caching test (~32GB min) is time consuming. I use ext2 instead of xfs (what I will be using), so that I could eliminate any effects of writing the journal from latency timings. XFS results in better read, write bandwidth and non-write latencies. Also, note that I've found that bonnie++ 1.93c does not show per char K/sec correctly(?), version 1.03 does, and I used 1.93c below.


Code:

Anticipatory, ext2

Tiotest results for 4 concurrent io threads:
,----------------------------------------------------------------------.
| Item                  | Time     | Rate         | Usr CPU  | Sys CPU |
+-----------------------+----------+--------------+----------+---------+
| Write        8192 MBs |  152.7 s |  53.637 MB/s |   0.7 %  |  11.4 % |
| Random Write   16 MBs |    4.1 s |   3.818 MB/s |   0.1 %  |   1.3 % |
| Read         8192 MBs |  224.7 s |  36.454 MB/s |   0.4 %  |   5.1 % |
| Random Read    16 MBs |    7.4 s |   2.102 MB/s |   0.1 %  |   0.4 % |
`----------------------------------------------------------------------'
Tiotest latency results:
,-------------------------------------------------------------------------.
| Item         | Average latency | Maximum latency | % >2 sec | % >10 sec |
+--------------+-----------------+-----------------+----------+-----------+
| Write        |        0.196 ms |     1137.279 ms |  0.00000 |   0.00000 |
| Random Write |        2.395 ms |      343.993 ms |  0.00000 |   0.00000 |
| Read         |        0.424 ms |      537.257 ms |  0.00000 |   0.00000 |
| Random Read  |        6.895 ms |      104.739 ms |  0.00000 |   0.00000 |
|--------------+-----------------+-----------------+----------+-----------|
| Total        |        0.318 ms |     1137.279 ms |  0.00000 |   0.00000 |
`--------------+-----------------+-----------------+----------+-----------'

Tiotest results for 1 concurrent io threads:
,----------------------------------------------------------------------.
| Item                  | Time     | Rate         | Usr CPU  | Sys CPU |
+-----------------------+----------+--------------+----------+---------+
| Write        8192 MBs |  109.0 s |  75.176 MB/s |   0.7 %  |  13.1 % |
| Random Write    4 MBs |    1.3 s |   3.020 MB/s |   0.0 %  |   1.5 % |
| Read         8192 MBs |   42.5 s | 192.654 MB/s |   1.6 %  |  25.6 % |
| Random Read     4 MBs |    3.8 s |   1.029 MB/s |   0.1 %  |   0.2 % |
`----------------------------------------------------------------------'
Tiotest latency results:
,-------------------------------------------------------------------------.
| Item         | Average latency | Maximum latency | % >2 sec | % >10 sec |
+--------------+-----------------+-----------------+----------+-----------+
| Write        |        0.031 ms |    33147.868 ms |  0.00005 |   0.00005 |
| Random Write |        0.566 ms |        9.925 ms |  0.00000 |   0.00000 |
| Read         |        0.020 ms |       17.176 ms |  0.00000 |   0.00000 |
| Random Read  |        3.794 ms |       20.777 ms |  0.00000 |   0.00000 |
|--------------+-----------------+-----------------+----------+-----------|
| Total        |        0.027 ms |    33147.868 ms |  0.00002 |   0.00002 |
`--------------+-----------------+-----------------+----------+-----------'

Version 1.93c       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
taz              7G   920  99 77348  12 52209  10  1424  98 201612  25 792.5   6
Latency              9104us     670ms     630ms   23415us   26568us     548ms
Version 1.93c       ------Sequential Create------ --------Random Create--------
taz                 -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16  6846  94 +++++ +++ +++++ +++  7173  96 +++++ +++ 22568  93
Latency              9174us      14us      30us    5238us      12us     277us


Deadline, ext2

Tiotest results for 4 concurrent io threads:
,----------------------------------------------------------------------.
| Item                  | Time     | Rate         | Usr CPU  | Sys CPU |
+-----------------------+----------+--------------+----------+---------+
| Write        8192 MBs |  148.0 s |  55.339 MB/s |   0.7 %  |  11.4 % |
| Random Write   16 MBs |    3.6 s |   4.390 MB/s |   0.1 %  |   1.9 % |
| Read         8192 MBs |   84.2 s |  97.271 MB/s |   0.8 %  |  13.6 % |
| Random Read    16 MBs |    5.2 s |   3.010 MB/s |   0.1 %  |   0.7 % |
`----------------------------------------------------------------------'
Tiotest latency results:
,-------------------------------------------------------------------------.
| Item         | Average latency | Maximum latency | % >2 sec | % >10 sec |
+--------------+-----------------+-----------------+----------+-----------+
| Write        |        0.187 ms |      948.769 ms |  0.00000 |   0.00000 |
| Random Write |        1.733 ms |      239.241 ms |  0.00000 |   0.00000 |
| Read         |        0.155 ms |      138.513 ms |  0.00000 |   0.00000 |
| Random Read  |        3.972 ms |       22.511 ms |  0.00000 |   0.00000 |
|--------------+-----------------+-----------------+----------+-----------|
| Total        |        0.177 ms |      948.769 ms |  0.00000 |   0.00000 |
`--------------+-----------------+-----------------+----------+-----------'

Tiotest results for 1 concurrent io threads:
,----------------------------------------------------------------------.
| Item                  | Time     | Rate         | Usr CPU  | Sys CPU |
+-----------------------+----------+--------------+----------+---------+
| Write        8192 MBs |  112.6 s |  72.784 MB/s |   0.7 %  |  13.2 % |
| Random Write    4 MBs |    1.3 s |   3.082 MB/s |   0.2 %  |   0.8 % |
| Read         8192 MBs |   42.4 s | 193.102 MB/s |   1.6 %  |  24.3 % |
| Random Read     4 MBs |    4.0 s |   0.974 MB/s |   0.0 %  |   0.2 % |
`----------------------------------------------------------------------'
Tiotest latency results:
,-------------------------------------------------------------------------.
| Item         | Average latency | Maximum latency | % >2 sec | % >10 sec |
+--------------+-----------------+-----------------+----------+-----------+
| Write        |        0.032 ms |     1641.053 ms |  0.00000 |   0.00000 |
| Random Write |        0.504 ms |        9.520 ms |  0.00000 |   0.00000 |
| Read         |        0.020 ms |       16.559 ms |  0.00000 |   0.00000 |
| Random Read  |        4.011 ms |       31.399 ms |  0.00000 |   0.00000 |
|--------------+-----------------+-----------------+----------+-----------|
| Total        |        0.027 ms |     1641.053 ms |  0.00000 |   0.00000 |
`--------------+-----------------+-----------------+----------+-----------'

Version 1.93c       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
taz              7G  1055  98 74916  12 50356  10  1610  97 202775  24  1150   9
Latency              7936us   35201ms     383ms   22374us   25876us     589ms
Version 1.93c       ------Sequential Create------ --------Random Create--------
taz                 -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16  6934  95 +++++ +++ +++++ +++  7149  97 +++++ +++ 22126  93
Latency              6253us      12us      56us    7395us      12us     288us

CFQ, ext2

Tiotest results for 4 concurrent io threads:
,----------------------------------------------------------------------.
| Item                  | Time     | Rate         | Usr CPU  | Sys CPU |
+-----------------------+----------+--------------+----------+---------+
| Write        8192 MBs |  133.8 s |  61.235 MB/s |   0.8 %  |  13.0 % |
| Random Write   16 MBs |    3.6 s |   4.388 MB/s |   0.0 %  |   1.6 % |
| Read         8192 MBs |   87.6 s |  93.490 MB/s |   0.8 %  |  13.2 % |
| Random Read    16 MBs |    6.5 s |   2.419 MB/s |   0.0 %  |   0.7 % |
`----------------------------------------------------------------------'
Tiotest latency results:
,-------------------------------------------------------------------------.
| Item         | Average latency | Maximum latency | % >2 sec | % >10 sec |
+--------------+-----------------+-----------------+----------+-----------+
| Write        |        0.145 ms |    13981.799 ms |  0.00129 |   0.00038 |
| Random Write |        1.980 ms |      499.414 ms |  0.00000 |   0.00000 |
| Read         |        0.148 ms |      124.167 ms |  0.00000 |   0.00000 |
| Random Read  |        3.931 ms |       92.474 ms |  0.00000 |   0.00000 |
|--------------+-----------------+-----------------+----------+-----------|
| Total        |        0.152 ms |    13981.799 ms |  0.00064 |   0.00019 |
`--------------+-----------------+-----------------+----------+-----------'

Tiotest results for 1 concurrent io threads:
,----------------------------------------------------------------------.
| Item                  | Time     | Rate         | Usr CPU  | Sys CPU |
+-----------------------+----------+--------------+----------+---------+
| Write        8192 MBs |   90.0 s |  90.997 MB/s |   0.9 %  |  17.1 % |
| Random Write    4 MBs |    1.8 s |   2.156 MB/s |   0.0 %  |   0.9 % |
| Read         8192 MBs |   42.1 s | 194.630 MB/s |   1.7 %  |  25.8 % |
| Random Read     4 MBs |    3.9 s |   1.014 MB/s |   0.1 %  |   0.2 % |

`----------------------------------------------------------------------'
Tiotest latency results:
,-------------------------------------------------------------------------.
| Item         | Average latency | Maximum latency | % >2 sec | % >10 sec |
+--------------+-----------------+-----------------+----------+-----------+
| Write        |        0.029 ms |    16312.277 ms |  0.00005 |   0.00005 |
| Random Write |        1.043 ms |        9.102 ms |  0.00000 |   0.00000 |
| Read         |        0.020 ms |       19.132 ms |  0.00000 |   0.00000 |
| Random Read  |        3.852 ms |       27.849 ms |  0.00000 |   0.00000 |
|--------------+-----------------+-----------------+----------+-----------|
| Total        |        0.026 ms |    16312.277 ms |  0.00002 |   0.00002 |
`--------------+-----------------+-----------------+----------+-----------'

Version 1.93c       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
taz              7G   911  99 84611  16 50582  11  1313  98 199747  25  1131  10
Latency             10157us   16763ms     289ms   24113us   24422us     589ms
Version 1.93c       ------Sequential Create------ --------Random Create--------
taz                 -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16  6039  82 +++++ +++ +++++ +++  6171  84 +++++ +++ 21658  91
Latency              7275us      17us      30us    6350us      19us     231us


_________________
Calling fly fishing a hobbie is like calling brain surgery a job.
Back to top
View user's profile Send private message
commonloon
Tux's lil' helper
Tux's lil' helper


Joined: 13 Apr 2004
Posts: 133
Location: Marin, CA

PostPosted: Mon Feb 14, 2005 2:46 am    Post subject: Reply with quote

This came across LKML. While I haven't read it completely, it seems to have a nice overview of the O(1), etc. scheduler.

http://josh.trancesoftware.com/linux/
_________________
Calling fly fishing a hobbie is like calling brain surgery a job.
Back to top
View user's profile Send private message
bollucks
l33t
l33t


Joined: 27 Oct 2004
Posts: 606

PostPosted: Mon Feb 14, 2005 3:56 am    Post subject: Reply with quote

steve_d555 wrote:
I have a question, I hear that Deadline/Anticipatory is better for gaming as it allows it to take up 100%. Does anyone have any thoughts? Has anyone done any benchmarks against CFQ or something? I would be really interested to know, because ive been using Nitro sources with CFQ and was wondering what was better.

There's no such thing as "not allowing 100%". They all (cpu and i/o schedulers) try to use all the resources as much as possible, as efficiently as possible. It makes no sense to have cpu or i/o bandwidth available and not use it. The closest any of these schedulers comes to not using 100% is the anticipatory I/O scheduler which pauses briefly after a read (up to 6ms) anticipating another read because it would be cheaper to perform a related read after a short wait than starting a write or different read. It still does this to try and maximise throughput. None of them try to decrease throughput in any way, but they definitely do perform differently in different workloads.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum