| 00:11.58 | *** join/#gllug Leeds (~richardc@n219078058222.netvigator.com) |
| 00:11.58 | *** join/#gllug sabinef72 (~sabinef72@barcelone.ipv6.popipo.fr) |
| 00:11.58 | *** join/#gllug BigRedS (~avi@catflap.positive-internet.com) |
| 00:11.58 | *** join/#gllug kjs (~kjs@fedora/kjs) |
| 00:21.51 | *** join/#gllug sabinef72 (~sabinef72@barcelone.ipv6.popipo.fr) |
| 00:34.02 | *** join/#gllug Leeds (~richardc@n219078058222.netvigator.com) |
| 00:49.37 | *** join/#gllug sabinef72 (~sabinef72@barcelone.ipv6.popipo.fr) |
| 01:22.54 | *** join/#gllug sabinef72 (~sabinef72@barcelone.ipv6.popipo.fr) |
| 01:22.54 | *** join/#gllug BigRedS (~avi@catflap.positive-internet.com) |
| 01:22.54 | *** join/#gllug kjs (~kjs@fedora/kjs) |
| 01:25.14 | *** join/#gllug color_machine (~chatzilla@118.69.217.210) |
| 01:51.50 | *** join/#gllug sabinef72 (~sabinef72@barcelone.ipv6.popipo.fr) |
| 02:13.23 | *** join/#gllug color_machine (~chatzilla@118.69.217.210) |
| 02:14.37 | *** join/#gllug color_machine (~chatzilla@118.69.217.210) |
| 02:58.11 | *** join/#gllug Leeds (~richardc@www.scorefive.com) |
| 05:01.28 | *** join/#gllug boudiccas (~boudiccas@78.146.161.192) |
| 06:15.52 | *** join/#gllug DiscordianUK (~ch@fedora/DiscordianUK) |
| 07:31.11 | j0nr | morning |
| 08:11.24 | *** join/#gllug Leeds (~richardc@n219078058222.netvigator.com) |
| 09:08.03 | ChoHag | Periodically, bring a few key IT brains to the boardroom to observe the problems of the organization at large, even about things outside of the IT world, if only to make use of their exquisitely refined BS detectors. |
| 09:17.47 | *** join/#gllug DiscordianUK (~ch@fedora/DiscordianUK) |
| 10:19.12 | *** join/#gllug stu_ (~stu@dyn1240-126.vpn.ic.ac.uk) |
| 12:38.08 | *** join/#gllug dick_turpin (~dick_turp@host217-34-163-30.in-addr.btopenworld.com) |
| 12:39.34 | dick_turpin | Afternoon all |
| 13:02.58 | boudiccas | evening all! its a nice day to be a copper, good criminal catching weather |
| 14:59.25 | *** join/#gllug Leeds (~richardc@n219078058222.netvigator.com) |
| 15:24.02 | mindbendr | could someone explain how this block sizes work in the application level |
| 15:24.18 | mindbendr | when you have ext3 with 4K blocksize |
| 15:24.37 | mindbendr | how a certain application pick a block size when it decides to do read/writes? |
| 15:24.57 | wethrin | that would be an abstraction violation |
| 15:25.02 | mindbendr | as in mysql having 1K block sizes for myisam |
| 15:25.12 | wethrin | the application should not care what filesystem is underneath |
| 15:25.51 | mindbendr | so the kernel is breaking down the blocks into pieces |
| 15:26.03 | mindbendr | then writes as 4K on ext3 |
| 15:26.23 | wethrin | if each myqsl block is a single file.... |
| 15:26.24 | mindbendr | but why certain speed tests provide different results when you define the block size like 1Mb instead 4k |
| 15:26.30 | wethrin | which it's not, because that'd be hideously inefficient |
| 15:26.57 | mindbendr | so how does it work |
| 15:27.15 | wethrin | Magic(TM) |
| 15:27.49 | wethrin | there's an optimimum amount of data that can be transferred in one chunk before requiring to wait for any acknowledgements |
| 15:31.35 | mindbendr | so why does it different if you change the blocksize from 4k to 64K even though the filesystem has been set to 4k |
| 15:31.42 | mindbendr | *why does it differ |
| 15:33.25 | wethrin | filesystems will do lots of forms of caching and readaheads |
| 15:34.25 | mindbendr | yeah I understand that |
| 15:34.49 | mindbendr | the thing is, I've been running various iometer tests on raid6 |
| 15:35.00 | mindbendr | with 6disksx15RPM |
| 15:35.39 | *** part/#gllug dick_turpin (~dick_turp@host217-34-163-30.in-addr.btopenworld.com) |
| 15:36.18 | mindbendr | and it's super slow on 100% random reads (like 2 Mb/sec) |
| 15:37.10 | mindbendr | but when the blocksize is 1M, the sequential read goes up to 800Mb/sec |
| 15:37.53 | mindbendr | assuming it hasn't cached the raw disk (no fs etc) :) |
| 15:38.08 | mindbendr | how does it go up to 800Mb whereas when you limit it to 4K it's slow |
| 15:38.50 | mindbendr | even when you do dd if=file.img of=/logical/volume bs=1Mb works faster than bs=4k |
| 15:39.34 | mindbendr | if the file system has been set to 4k, how can it fetch the data more than 4k? as it can't why does it read the data faster (assuming there's no caching going on for a 50GB image) |
| 15:40.57 | wethrin | you can fetch multiple blocks at a time |
| 15:41.51 | mindbendr | how? |
| 15:42.32 | wethrin | taken care of by internal drive electronics, device drivers, etc. |
| 15:44.58 | mindbendr | limited with the number of heads on the drive? |
| 15:46.32 | *** join/#gllug reodge (~mark@94-194-51-173.zone8.bethere.co.uk) |
| 15:46.40 | wethrin | drives tend to have two heads per platter |
| 15:47.45 | mindbendr | so if you are going buy a disk unit (SAN or JBOD) |
| 15:48.08 | mindbendr | would you ignore the values they provide in various blocksizes larger than 4K if you're going to use ext3 |
| 15:48.42 | wethrin | Mu. Filesystems are still abstracted from the underlying structure of the hard disk |
| 15:48.43 | *** join/#gllug stu__ (~stu@dyn1240-126.vpn.ic.ac.uk) |
| 15:48.51 | mindbendr | because essentially, if you are going to run an OS and DB, there's not much point of considering larger than 4K blocksizes |
| 15:50.30 | wethrin | a blocksize on a filesystem is merely a minimum allocation unit size |
| 15:50.34 | mindbendr | but on the application, it's using 4K bs, so why would I care how would it perform if i'd like to fetch data with 1M bs in general |
| 15:50.49 | antiphase | Not much use having block size that's larger than the page size if you want to use it as virtual memory |
| 15:51.02 | antiphase | Hence 4k |
| 15:51.36 | mindbendr | use as virtual memory? |
| 15:52.26 | antiphase | The 4k block size limit for all Linux file systems is because of the kernel page size |
| 15:52.44 | antiphase | The ability to map files into memory is also very important |
| 15:52.58 | mindbendr | ah |
| 15:52.59 | antiphase | and it doesn't seem to impede things, on the whole |
| 15:53.38 | mindbendr | antiphase: so simply, would you care about the speed results of a benchmark that has been done in large blocksizes? |
| 15:53.55 | mindbendr | antiphase: because it seems they don't perform very different if bs=4K |
| 15:54.16 | antiphase | That's to do with the I/O layer, not the file system. You're simply requiring more system calls to move the same amount of data |
| 15:55.09 | mindbendr | so it's IO ps the hardware can handle |
| 15:55.12 | antiphase | if you set bs=1 it will be very slow. Beyond 4k it's probably not going to change much |
| 15:55.21 | mindbendr | it does though :( |
| 15:55.33 | mindbendr | there's a raid unit called dot hills |
| 15:55.35 | antiphase | Well not as much as going from 1 to 4k :P |
| 15:55.43 | mindbendr | it's 10.5K pounds |
| 15:56.17 | antiphase | 10.5 kilo pounds? Great Scott! |
| 15:56.21 | mindbendr | IO bs=4k , you get 15Mb/sec whereas |
| 15:56.34 | antiphase | Do you mean Mb or MB? |
| 15:56.43 | mindbendr | bs=1M, the speed is 396M/sec |
| 15:56.55 | mindbendr | random read |
| 15:57.13 | antiphase | That sounds like some good caching |
| 15:57.29 | antiphase | When you say bs=1M it makes me think of dd |
| 15:57.30 | mindbendr | on raw device? |
| 15:57.39 | mindbendr | no filesystem involvement |
| 15:58.17 | antiphase | You shouldn't expect much from your storage if it was only £10.5k |
| 15:58.38 | mindbendr | yeah but |
| 15:58.51 | antiphase | I doubt you're managing to randomly read 396 1MB blocks per second |
| 15:58.52 | mindbendr | you get similar results with a 5K server including the controller |
| 15:59.00 | mindbendr | why not |
| 15:59.07 | mindbendr | in 1TB disk |
| 15:59.14 | antiphase | You also don't specify the concurrency |
| 15:59.35 | antiphase | or how you're measuring it. |
| 15:59.46 | antiphase | I'd get hold of a decent tool, if you're not already using one |
| 15:59.54 | mindbendr | 1 worker with iometer |
| 16:00.34 | mindbendr | antiphase: essentially, the problem is running 14 VMs on 6 disks with raid6 (15.000 RPM) |
| 16:00.43 | antiphase | Oh dear |
| 16:00.51 | mindbendr | they are mostly idle :) |
| 16:01.07 | mindbendr | but if someone starts copying a file, thanks to rubbish Xen I/O utilisation |
| 16:01.11 | antiphase | You might as well have used 1+0 if you're wasting 2/6 on parity |
| 16:01.16 | mindbendr | the rest of the systems pause |
| 16:01.44 | mindbendr | but the storage provider asked how much I/O per sec do I get by running i/o meter |
| 16:01.58 | antiphase | tbh you've got effectively 1/4 disk per server, so any thought of I/O intensive stuff is clearly out of the question |
| 16:02.03 | mindbendr | it turns out 300 IO per sec * 4096 bytes on 100% random |
| 16:02.30 | mindbendr | 1/4 disk per server if it's not idle though, if one them is actively used |
| 16:02.34 | antiphase | 300 IOPS is what you could expect out of a single disk, and I expect you're wasting most of them on parity writes |
| 16:02.48 | mindbendr | hmm |
| 16:02.56 | antiphase | parity RAId really isn't that good if you're expecting throughput |
| 16:03.23 | antiphase | Ha, it's beer o'clock here anyway |
| 16:03.36 | mindbendr | :) |
| 16:32.22 | *** join/#gllug DiscordianUK (~ch@fedora/DiscordianUK) |
| 16:39.48 | *** join/#gllug Armand (~me@host86-180-187-231.range86-180.btcentralplus.com) |
| 20:30.02 | *** join/#gllug Armand (~me@host86-180-187-231.range86-180.btcentralplus.com) |
| 21:00.38 | *** join/#gllug Armand (~me@host86-180-187-231.range86-180.btcentralplus.com) |
| 21:15.25 | *** join/#gllug AndyMillar (~andy-free@2001:ba8:0:1c7::d40d:c98c) |
| 22:30.31 | *** join/#gllug Discordian (~ch@fedora/DiscordianUK) |
| 22:41.39 | *** join/#gllug bilarh (~henrik@128-177-27-125.ip.openhosting.com) |
| 22:41.39 | *** join/#gllug wethrin (dankolb@hades.eco.li) |
| 22:41.39 | *** join/#gllug Dominic (~dominic@chaldene.m0dlx.com) |
| 22:41.39 | *** join/#gllug antiphase (~ant@plasma.ossified.net) |
| 22:41.39 | *** join/#gllug Nafallo (~nafallo@ubuntu/member/nafallo) |
| 22:41.39 | *** join/#gllug ChoHag (~mking@109-170-148-201.xdsl.murphx.net) |
| 22:41.39 | *** join/#gllug reodge (~mark@94-194-51-173.zone8.bethere.co.uk) |
| 22:41.39 | *** join/#gllug AndyMillar (~andy-free@2001:ba8:0:1c7::d40d:c98c) |
| 22:41.39 | *** join/#gllug sabinef72 (~sabinef72@barcelone.ipv6.popipo.fr) |
| 22:41.39 | *** join/#gllug BigRedS (~avi@catflap.positive-internet.com) |
| 22:41.39 | *** join/#gllug kjs (~kjs@fedora/kjs) |
| 22:53.38 | *** join/#gllug DiscordianUK (~ch@fedora/DiscordianUK) |