1. Info about what was wrong
all sizes are in SECTORS=512 bytes 312500000 sda (same layout for sdb) 1959866 sda1 15631244 sda2 294905204 sda3 15631104 md0 294904960 md1 1959864 sda1 filesystem size 294904960 md1 filesystem size = 36863120 4096-byte blocks 312496135 sector causing I/O error write (sda and sdb) 294904960 sector causing I/O error, offset from sd?3 start. 256 sectors for raid superblock (at end of device) partition table for both sda and sdb: Device Boot Start End Blocks Id System /dev/sda1 63 1959929 979933+ 83 Linux /dev/sda2 1959930 17591174 7815622+ fd Linux raid autodetect /dev/sda3 17591175 312496379 147452602+ fd Linux raid autodetect === to do == 294904960 sector causing I/O error, offset from sd?3 start. 294904896 new target partition size (not actually going to resize) 294904640 new target filesystem size (partition size minus 256-sector superblock) 294904384 new target filesystem size (partition size minus 512 sectors for safety) 36863048 new filesystem size, in sectors (part-512sectors) 294904704 sector minimum filesystem size to create. resize2fs mdadm --grow
2. actually done to fix it this day
fsck -f /dev/md1 resize2fs /dev/md1 -p 36861840 # resized to 36861840 4Kblocks, 294894720 sectors, 147447360 KiB mdadm --grow --size=147447360 /dev/md1 resize partition to 294894720 + 512 sectors = 294895232 sectors = 147447616 KiB fdisk /dev/sdb >>> u >>> d 3 >>> n p 3 \n +294895232 mdadm --add /dev/md1 /dev/sdb3 #NOTE a=b, b=3 in the live system (wait for sync) mdadm --fail /dev/md1 /dev/sdc3 mdadm --remove /dev/md1 /dev/sdc3 fdisk /dev/sd3 >>> <same> mdadm --add /dev/md1 /dev/sdc3 (wait for sync) reboot, and it works! == New layout and stuff == Device Boot Start End Blocks Id System /dev/sda1 63 1959929 979933+ 83 Linux /dev/sda2 1959930 17591174 7815622+ fd Linux raid autodetect /dev/sda3 17591175 312486407 147447616+ 83 Linux New sector error: end_request: I/O error, dev sda, sector 312486279