I've added the nvme disk support, and have tested it with nvme disk in virtual box, and it seems to work fine, but recently have a user that has an issue where the disk image doesn't seem to get compressed at all. Has a 256G disk, and the image is about the same size?? In my tests, I have a 4096M nvme disk, and two about 2048M partitions. Partitions are cleared. Image of disk results in about 30K file, and about 15K for each of the partitions. Which is what I would expect. His disk is on a new machine and has Windows 10?
Have newer working 0.54 copy using Fedora 27 as a build, and also has the latest kernels. So, looking for someone with experience with the nvme solid state disks. Not sure if it is some kind of encryption or what it could be. Thanks.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The issue seems to have been resolved. Seems the user was using the program Eraser to clear the partitions, and it uses a default process of leaving randomized data in the unused space as a security measure, which is great from a security option, but results in super bad compress of bit level images. After using the G4L options to clear the space that fills it will nulls, the image resulted in the expected high compression ratio. Interesting process to discover the real issue.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I recently imaged an 128GB NVMe drive.
It was a project where I did a number of partition deletes & installs of windows versions (Embedded 7 through 8.1) along with copying a 10gb game to it, over and over.
Doing an raw-mode whole-drive image was resulting in a compressed image over 50GB, even though the 119GB NTFS partition contained only 26GB uncompressed data, via ntfsbackup.
Finding that a silly wasteful size, I thought about it for a while, and surmised that the raw read was preserving all the bits of the long-deleted data from who-know-when.
Taking a chance, I picked 'choose local mount' from the menu, and selected the NTFS partition on the NVMe, then exited to the console.
Doing an 'mount' I saw it mounted to '/mnt/local' as expected, and I did this:
'dd if=/dev/zero of=/mnt/local/big_zero.file bs=1M status=progress'
And watched the file grow to take up every bit of free space on the mounted NTFS volume.
After that was done, I deleted the 'big_zero.file', and went back into g4l.
Imaging the whole-drive to my ftp server again afterward; Resulted in an image now happily 16GB compressed.
Perhaps such an 'Zero empty space within (supported) partitions' option could be added to the menu, with a disclaimer maybe?
Though that would touch every block of the ssd drive with a single write, as well as remove any possibility of 'un-deleting' files later, I find it a worthy sacrifice for faster imaging/smaller images. SSD-wise, it would be like adding a single mile to a cars wear & tear?
PS. the NVMe disks performance is still excellent afterwards
Last edit: Christopher Baker 2024-12-21
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Is under the Utility option for zeroing out the space. Generally can zero out various partition types. Saw the same thing long ago. Had done a clean install of Fedora Core 4 on an 80G disk, and made an image and it was much larger than I expected. Think it was like 24G. Then cleared the unused sectors, and redid the image, and it was like 2.8G. The raw mode does copy all raw sectors, so if they contain random data it takes up space to compress them. If they have been filled with 0's the compression program reduces size to very small amount. Has been part of G4L for long time. Does make a big difference, but sometimes takes a while to clean all the space with newer big disks. The NTFSCLONE option only backs up used data on NTFS partitions, but it does require the partitions already exist. In my classroom machines, had a partition that contained an NTFSCLONE image of the windows partition, and option on grub menu that would restore it in about 10 minutes to fix any machines locally. Also the udpcast option allowed me to image one machine to the other 19 at one time.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Brilliant! Just tried it out, and yup 'Zero Tool' under Utilities. How did I not see that?
I probably mistakenly got it into my head that it was some kind of pre-OS-installation partition clearing tool...
Shows that even with OS's & SSD's TRIM function and firmwares, abandoned data still lives on these drives like the rotating spindle drives do.
Ah, at least our posts might inform some googlers out there to the use & benefits of the menu option?
Thanks Mike!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I've added the nvme disk support, and have tested it with nvme disk in virtual box, and it seems to work fine, but recently have a user that has an issue where the disk image doesn't seem to get compressed at all. Has a 256G disk, and the image is about the same size?? In my tests, I have a 4096M nvme disk, and two about 2048M partitions. Partitions are cleared. Image of disk results in about 30K file, and about 15K for each of the partitions. Which is what I would expect. His disk is on a new machine and has Windows 10?
Have newer working 0.54 copy using Fedora 27 as a build, and also has the latest kernels. So, looking for someone with experience with the nvme solid state disks. Not sure if it is some kind of encryption or what it could be. Thanks.
The issue seems to have been resolved. Seems the user was using the program Eraser to clear the partitions, and it uses a default process of leaving randomized data in the unused space as a security measure, which is great from a security option, but results in super bad compress of bit level images. After using the G4L options to clear the space that fills it will nulls, the image resulted in the expected high compression ratio. Interesting process to discover the real issue.
I recently imaged an 128GB NVMe drive.
It was a project where I did a number of partition deletes & installs of windows versions (Embedded 7 through 8.1) along with copying a 10gb game to it, over and over.
Doing an raw-mode whole-drive image was resulting in a compressed image over 50GB, even though the 119GB NTFS partition contained only 26GB uncompressed data, via ntfsbackup.
Finding that a silly wasteful size, I thought about it for a while, and surmised that the raw read was preserving all the bits of the long-deleted data from who-know-when.
Taking a chance, I picked 'choose local mount' from the menu, and selected the NTFS partition on the NVMe, then exited to the console.
Doing an 'mount' I saw it mounted to '/mnt/local' as expected, and I did this:
'dd if=/dev/zero of=/mnt/local/big_zero.file bs=1M status=progress'
And watched the file grow to take up every bit of free space on the mounted NTFS volume.
After that was done, I deleted the 'big_zero.file', and went back into g4l.
Imaging the whole-drive to my ftp server again afterward; Resulted in an image now happily 16GB compressed.
Perhaps such an 'Zero empty space within (supported) partitions' option could be added to the menu, with a disclaimer maybe?
Though that would touch every block of the ssd drive with a single write, as well as remove any possibility of 'un-deleting' files later, I find it a worthy sacrifice for faster imaging/smaller images. SSD-wise, it would be like adding a single mile to a cars wear & tear?
PS. the NVMe disks performance is still excellent afterwards
Last edit: Christopher Baker 2024-12-21
Is under the Utility option for zeroing out the space. Generally can zero out various partition types. Saw the same thing long ago. Had done a clean install of Fedora Core 4 on an 80G disk, and made an image and it was much larger than I expected. Think it was like 24G. Then cleared the unused sectors, and redid the image, and it was like 2.8G. The raw mode does copy all raw sectors, so if they contain random data it takes up space to compress them. If they have been filled with 0's the compression program reduces size to very small amount. Has been part of G4L for long time. Does make a big difference, but sometimes takes a while to clean all the space with newer big disks. The NTFSCLONE option only backs up used data on NTFS partitions, but it does require the partitions already exist. In my classroom machines, had a partition that contained an NTFSCLONE image of the windows partition, and option on grub menu that would restore it in about 10 minutes to fix any machines locally. Also the udpcast option allowed me to image one machine to the other 19 at one time.
Brilliant! Just tried it out, and yup 'Zero Tool' under Utilities. How did I not see that?
I probably mistakenly got it into my head that it was some kind of pre-OS-installation partition clearing tool...
Shows that even with OS's & SSD's TRIM function and firmwares, abandoned data still lives on these drives like the rotating spindle drives do.
Ah, at least our posts might inform some googlers out there to the use & benefits of the menu option?
Thanks Mike!