2009/10/29

Using a loopback device to create arbitrary local volumes is convenient
but slow. Roughly twich as slow compared to a normal raid volume. Another
problem with that is that it is erratic. We have not explained this.
We tested a 35 gigs file turned into a volume using the loopback device.
Sometime, the throughput goes down to 20megabytes per seconds.

We have enhanced the tool "pattern" so it shows stats while it is running.

2009/10/28

Problemes when testing using gigabit ethernet. Wrong wires. The
pattern program was changed to use O_DIRECT (option -d) so we can
correctly mesure performance without the impact of the buffer cache.

2009/10/27

We have debugged the snapfile management. Corruption was created when
a read request was spanning two blocks passed 4gigs: unsigned overflow.

2009/10/21

Now that drsync is working, we are testing with real database with real
applications (loader files). So far, here are some results

One problem remains: Management of the multiple local volume. Each
local volume is paired with a drsnap remove volume in raid1 with
the --write-mostly option (on remote volume). A local volume is a device.
The remote drsnap volumes are plain files (big sparse files actually).
We need a way to manage the disk space locally. One solution is to use
LVM. Unfortunatly, for reason we don't understand yet, LVM is slow.
Another solution, less flexible is to use partitionnable raid volume.
Since kernel 2.6.26, all raid volume may be partitionned. This allows
us to build a raid10 (current configuration for our database server)
and split it many smaller devices. This is not as flexible as LVM since
it become tedious to resize a volume later.

For now, we did some experiments with loopback devices. So we end up
with a stack: raid10, xfs, big file, loopback , raid1 with nbd, xfs.
Probably highly innefficient. 

Here are some stats. On a 30 gigs database, running one day of loader
files resulted in 6 gigs of changes on drsnap using 1meg block. Performance
of the loader is roughly 50% slower. At this point, we don't know if
this is related to the loopback setup. Probably.

2009/10/21

I should have written this log before. At this point, the prototype
is working. A year ago I have done some tests, but was lacking a way
to jump between snapshots and synchronise the local volume.