Sometimes, one runs out of disk space on a virtual machine, and realizes that it was a mistake to provide such a small disk to it in the first place. Fortunately, unlike real disks, the virtual ones can be resized at will. A handy command for this task comes with QEMU (and, if you are on Linux, why are you using anything else?). Here is how to extend a raw disk image to 10 GB:
After running this command, the beginning of the disk will contain the old bytes that were there before, and at the end there will be a long run of zeroes. qemu-img is smart enough to avoid actually writing these zeros to the disk image, it creates a sparse file instead.
Resizing the disk image is only one-third of the job. The partition table still lists partitions of the old sizes, and the end of the disk is unused. Traditionally, fdisk has been the tool for altering the partition table. You can run it either from within your virtual machine, or directly on the disk image. All that is needed is to delete the last partition, and then recreate it with the same start sector, but with the correct size, so that it also covers the new part of the disk. Here is an example session with a simple MBR-based disk with two partitions:
As you see, it went smoothly. The kernel will pick up the new partition table after a reboot, and then you will be able to resize the filesystem with resize2fs (or some other tool if you are not using ext4).
Things are not so simple if the virtual disk is partitioned with GPT, not MBR, to begin with. The complication stems from the fact that there is a backup copy of GPT at the end of the disk. When we added zeros to the end of the disk, the backup copy ended up in the middle of the disk, not to be found. Also, the protective MBR now covers only the first part of the disk. The kernel is able to deal with this, but some versions of fdisk (at least fdisk found in Ubuntu 18.04) cannot. What happens is that fdisk is not able to create partitions that extend beyond the end of the old disk. And saving anything (in fact, even saving what already exists) fails with a rather unhelpful error message:
Modern versions of fdisk do not have this problem:
Still, with GPT and not-so-recent versions of fdisk, it looks like we cannot use fdisk to take advantage of the newly added disk space. There is another tool, gdisk, that can manipulate GPT structures. However, it claims that there is almost no usable free space on the disk, and thus refuses to usefully resize the last partition by default.
What we need to do is to use the "expert" functionality in order to move the backup GPT to the end of the disk. After that, new free space will be available, and we will be able to resize the last partition.
OK, it worked, but was a bit too complicated, to the degree sufficient to consider sticking with MBR where possible. And it changed the UUID of the last partition, which may or may not be OK in your setup (it is definitely not OK if /etc/fstab or the kernel command line mentions PARTUUID). The same warning about PARTUUID applies to modern versions of fdisk, too.
Anyway, it turns out that gdisk is not the simplest solution to the problem of resizing a GPT-based disk. The "sfdisk" program that comes with util-linux (i.e. with the same package that provides fdisk, even with not-so-recent versions) works just as well. We need to dump the existing partitions, edit the resulting script, and feed it back to sfdisk so that it recreates these partitions for us from scratch, with the correct sizes, and we can preserve all partition UUIDs, too.
Here is what this dump looks like:
We need to fix the "last-lba" parameter and change the size of the last partition. Also, newer versions of sfdisk add the "sector-size: 512" line that they don't understand. Fortunately, sfdisk has reasonable defaults (use as much space as possible) for all three parameters, so we can just delete them instead. Quite easy to do with sed:
Then, with some flags to turn off various checks, sfdisk loads the modified partition table dump:
Let me repeat. The following lines, ready to be copy-pasted without much thinking except for the disk name, resize the last partition to occupy as much disk space as possible, and they work both on GPT and MBR:
On Ubuntu, there is also a "cloud-guest-utils" package that provides an even-easier "growpart" command, here is how it works:
Just as with MBR, after resizing the partition, you have to reboot your virtual machine, so that the kernel picks up the new partition size, and then you can resize the filesystem to match the partition size.
qemu-img resize -f raw vda.img 10G
After running this command, the beginning of the disk will contain the old bytes that were there before, and at the end there will be a long run of zeroes. qemu-img is smart enough to avoid actually writing these zeros to the disk image, it creates a sparse file instead.
Resizing the disk image is only one-third of the job. The partition table still lists partitions of the old sizes, and the end of the disk is unused. Traditionally, fdisk has been the tool for altering the partition table. You can run it either from within your virtual machine, or directly on the disk image. All that is needed is to delete the last partition, and then recreate it with the same start sector, but with the correct size, so that it also covers the new part of the disk. Here is an example session with a simple MBR-based disk with two partitions:
# fdisk /dev/vda
Welcome to fdisk (util-linux 2.31.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
Command (m for help): p
Disk /dev/vda: 10 GiB, 10737418240 bytes, 20971520 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xc02e3411
Device Boot Start End Sectors Size Id Type
/dev/vda1 * 2048 997375 995328 486M 83 Linux
/dev/vda2 997376 12580863 11583488 5.5G 83 Linux
Command (m for help): d
Partition number (1,2, default 2): 2
Partition 2 has been deleted.
Command (m for help): n
Partition type
p primary (1 primary, 0 extended, 3 free)
e extended (container for logical partitions)
Select (default p): p
Partition number (2-4, default 2): 2
First sector (997376-20971519, default 997376):
Last sector, +sectors or +size{K,M,G,T,P} (997376-20971519, default 20971519):
Created a new partition 2 of type 'Linux' and of size 9.5 GiB.
Partition #2 contains a ext4 signature.
Do you want to remove the signature? [Y]es/[N]o: n
Command (m for help): p
Disk /dev/vda: 10 GiB, 10737418240 bytes, 20971520 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xc02e3411
Device Boot Start End Sectors Size Id Type
/dev/vda1 * 2048 997375 995328 486M 83 Linux
/dev/vda2 997376 20971519 19974144 9.5G 83 Linux
Command (m for help): w
The partition table has been altered.
Syncing disks.
As you see, it went smoothly. The kernel will pick up the new partition table after a reboot, and then you will be able to resize the filesystem with resize2fs (or some other tool if you are not using ext4).
Things are not so simple if the virtual disk is partitioned with GPT, not MBR, to begin with. The complication stems from the fact that there is a backup copy of GPT at the end of the disk. When we added zeros to the end of the disk, the backup copy ended up in the middle of the disk, not to be found. Also, the protective MBR now covers only the first part of the disk. The kernel is able to deal with this, but some versions of fdisk (at least fdisk found in Ubuntu 18.04) cannot. What happens is that fdisk is not able to create partitions that extend beyond the end of the old disk. And saving anything (in fact, even saving what already exists) fails with a rather unhelpful error message:
# fdisk /dev/vda Welcome to fdisk (util-linux 2.31.1). Changes will remain in memory only, until you decide to write them. Be careful before using the write command. GPT PMBR size mismatch (12582911 != 20971519) will be corrected by w(rite). GPT PMBR size mismatch (12582911 != 20971519) will be corrected by w(rite). Command (m for help): w GPT PMBR size mismatch (12582911 != 20971519) will be corrected by w(rite). fdisk: failed to write disklabel: Invalid argument
Modern versions of fdisk do not have this problem:
# fdisk /dev/vda Welcome to fdisk (util-linux 2.33). Changes will remain in memory only, until you decide to write them. Be careful before using the write command. GPT PMBR size mismatch (12582911 != 20971519) will be corrected by write. The backup GPT table is not on the end of the device. This problem will be corrected by write.
Still, with GPT and not-so-recent versions of fdisk, it looks like we cannot use fdisk to take advantage of the newly added disk space. There is another tool, gdisk, that can manipulate GPT structures. However, it claims that there is almost no usable free space on the disk, and thus refuses to usefully resize the last partition by default.
# gdisk /dev/vda GPT fdisk (gdisk) version 1.0.3 Partition table scan: MBR: protective BSD: not present APM: not present GPT: present Found valid GPT with protective MBR; using GPT. Command (? for help): p Disk /dev/vda: 20971520 sectors, 10.0 GiB Sector size (logical/physical): 512/512 bytes Disk identifier (GUID): 61E744EF-1CD3-5145-BC59-4646E6CB03DE Partition table holds up to 128 entries Main partition table begins at sector 2 and ends at sector 33 First usable sector is 2048, last usable sector is 12582878 Partitions will be aligned on 2048-sector boundaries Total free space is 2015 sectors (1007.5 KiB) Number Start (sector) End (sector) Size Code Name 1 2048 4095 1024.0 KiB EF02 2 4096 999423 486.0 MiB 8300 3 999424 12580863 5.5 GiB 8300
What we need to do is to use the "expert" functionality in order to move the backup GPT to the end of the disk. After that, new free space will be available, and we will be able to resize the last partition.
Command (? for help): x
Expert command (? for help): e
Relocating backup data structures to the end of the disk
Expert command (? for help): m
Command (? for help): d
Partition number (1-3): 3
Command (? for help): n
Partition number (3-128, default 3):
First sector (999424-20969472, default = 999424) or {+-}size{KMGTP}:
Last sector (999424-20969472, default = 20969472) or {+-}size{KMGTP}:
Current type is 'Linux filesystem'
Hex code or GUID (L to show codes, Enter = 8300):
Changed type of partition to 'Linux filesystem'
Command (? for help): p
Disk /dev/vda: 20971520 sectors, 10.0 GiB
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): 61E744EF-1CD3-5145-BC59-4646E6CB03DE
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 2048, last usable sector is 20969472
Partitions will be aligned on 2048-sector boundaries
Total free space is 0 sectors (0 bytes)
Number Start (sector) End (sector) Size Code Name
1 2048 4095 1024.0 KiB EF02
2 4096 999423 486.0 MiB 8300
3 999424 20969472 9.5 GiB 8300 Linux filesystem
Command (? for help): w
Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
PARTITIONS!!
Do you want to proceed? (Y/N): y
OK; writing new GUID partition table (GPT) to /dev/vda.
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot or after you
run partprobe(8) or kpartx(8)
The operation has completed successfully.
OK, it worked, but was a bit too complicated, to the degree sufficient to consider sticking with MBR where possible. And it changed the UUID of the last partition, which may or may not be OK in your setup (it is definitely not OK if /etc/fstab or the kernel command line mentions PARTUUID). The same warning about PARTUUID applies to modern versions of fdisk, too.
Anyway, it turns out that gdisk is not the simplest solution to the problem of resizing a GPT-based disk. The "sfdisk" program that comes with util-linux (i.e. with the same package that provides fdisk, even with not-so-recent versions) works just as well. We need to dump the existing partitions, edit the resulting script, and feed it back to sfdisk so that it recreates these partitions for us from scratch, with the correct sizes, and we can preserve all partition UUIDs, too.
Here is what this dump looks like:
# sfdisk --dump /dev/vda > disk.dump GPT PMBR size mismatch (12582911 != 20971519) will be corrected by w(rite). # cat disk.dump label: gpt label-id: 61E744EF-1CD3-5145-BC59-4646E6CB03DE device: /dev/vda unit: sectors first-lba: 2048 last-lba: 12582878 /dev/vda1 : start= 2048, size= 2048, type=[...], uuid=[...] /dev/vda2 : start= 4096, size= 995328, type=[...], uuid=[...] /dev/vda3 : start= 999424, size= 11581440, type=[...], uuid=[...]
We need to fix the "last-lba" parameter and change the size of the last partition. Also, newer versions of sfdisk add the "sector-size: 512" line that they don't understand. Fortunately, sfdisk has reasonable defaults (use as much space as possible) for all three parameters, so we can just delete them instead. Quite easy to do with sed:
# sed -i -e '/^sector-size:/d' -e '/^last-lba:/d' -e '$s/size=[^,]*,//' disk.dump
# cat disk.dump label: gpt label-id: 61E744EF-1CD3-5145-BC59-4646E6CB03DE device: /dev/vda unit: sectors first-lba: 2048 /dev/vda1 : start= 2048, size= 2048, type=[...], uuid=[...] /dev/vda2 : start= 4096, size= 995328, type=[...], uuid=[...] /dev/vda3 : start= 999424, type=[...], uuid=[...]
Then, with some flags to turn off various checks, sfdisk loads the modified partition table dump:
# sfdisk --no-reread --no-tell-kernel -f --wipe never /dev/vda < disk.dump GPT PMBR size mismatch (12582911 != 20971519) will be corrected by w(rite). Disk /dev/vda: 10 GiB, 10737418240 bytes, 20971520 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: gpt Disk identifier: 61E744EF-1CD3-5145-BC59-4646E6CB03DE Old situation: Device Start End Sectors Size Type /dev/vda1 2048 4095 2048 1M BIOS boot /dev/vda2 4096 999423 995328 486M Linux filesystem /dev/vda3 999424 12580863 11581440 5.5G Linux filesystem >>> Script header accepted. >>> Script header accepted. >>> Script header accepted. >>> Script header accepted. >>> Script header accepted. >>> Created a new GPT disklabel (GUID: 61E744EF-1CD3-5145-BC59-4646E6CB03DE). /dev/vda1: Created a new partition 1 of type 'BIOS boot' and of size 1 MiB. /dev/vda2: Created a new partition 2 of type 'Linux filesystem' and of size 486 MiB. Partition #2 contains a ext4 signature. /dev/vda3: Created a new partition 3 of type 'Linux filesystem' and of size 9.5 GiB. Partition #3 contains a ext4 signature. /dev/vda4: Done. New situation: Disklabel type: gpt Disk identifier: 61E744EF-1CD3-5145-BC59-4646E6CB03DE Device Start End Sectors Size Type /dev/vda1 2048 4095 2048 1M BIOS boot /dev/vda2 4096 999423 995328 486M Linux filesystem /dev/vda3 999424 20971486 19972063 9.5G Linux filesystem The partition table has been altered.
Let me repeat. The following lines, ready to be copy-pasted without much thinking except for the disk name, resize the last partition to occupy as much disk space as possible, and they work both on GPT and MBR:
DISK=/dev/vda sfdisk --dump $DISK > disk.dump sed -i -e '/^sector-size:/d' -e '/^last-lba:/d' -e '$s/size=[^,]*,//' disk.dump sfdisk --no-reread --no-tell-kernel -f --wipe never $DISK < disk.dump
On Ubuntu, there is also a "cloud-guest-utils" package that provides an even-easier "growpart" command, here is how it works:
# growpart /dev/vda 3 CHANGED: partition=3 start=999424 old: size=11581440 end=12580864 new: size=19972063,end=20971487
Just as with MBR, after resizing the partition, you have to reboot your virtual machine, so that the kernel picks up the new partition size, and then you can resize the filesystem to match the partition size.
No comments:
Post a Comment