Utility to Create, and Manage XXXXX.ucimg Disk partition Image Files

Moderator: Forum moderators

Post Reply
User avatar
rockedge
Site Admin
Posts: 5774
Joined: Mon Dec 02, 2019 1:38 am
Location: Connecticut,U.S.A.
Has thanked: 2052 times
Been thanked: 2134 times
Contact:

Utility to Create, and Manage XXXXX.ucimg Disk partition Image Files

Post by rockedge »

I am working with "sparse" files so the disk image will contain 0 bytes at creation and will show that with du -m upper_changes.ucimg
but with ls -l upper_changes.ucimg will show it's 1024 MiB. In effect is dynamic in storage sizes.

Code: Select all

dd if=/dev/zero of=upper_changes.ucimg count=102400
mkfs.ext3 upper_changes.ucimg

This example is the command used for the "sparse" image.

Code: Select all

dd if=/dev/zero of=upper_changes.ucimg bs=1M count=0 seek=1024
mkfs.ext3 upper_changes.ucimg
Screenshot_2022-12-12_13-21-01.png
Screenshot_2022-12-12_13-21-01.png (90.82 KiB) Viewed 588 times

To resize this :

Code: Select all

dd if=/dev/zero of=sparse.img bs=1M count=0 seek=2048

which now makes upper_changes.ucimg 2048 MiB. Now the partition needs to be expanded to fit the new size.

A utility that does this already exists in Puppy Linux.

User avatar
rockedge
Site Admin
Posts: 5774
Joined: Mon Dec 02, 2019 1:38 am
Location: Connecticut,U.S.A.
Has thanked: 2052 times
Been thanked: 2134 times
Contact:

Re: Utility to Create, and Manage XXXXX.ucimg Disk partition Image Files

Post by rockedge »

@wiak I set up a USB drive (8 GiB) as a small FAT32 boot partition and the rest as NTFS. I added KLV-Airedale-rc2 frugally to the NTFS partition. Used KLV to make the drive's partitions and setup the boot with Grub4Dos.

I made a 2 GiB upper_changes.ucimg using the commands above. Tried both sparse model and regular model and both work. I booted the system and made some changes because weirdly enough the desktop appeared like the stock version of KLV coming straight from the build scripts. So I rearranged the task bars and background and rebooted. Which was successful and all the changes are persistent

BUT the boot process is taking a really long time! Like at first it was so long I kept hard stopping the machine. Which is the old DELL VOSTRO 1500 single core CPU and 4 GiB of RAM that I just added. Then I booted and went to get a coffee. Well the machine booted finally and was up and running when I came back. Now I have measured the boot cycle time and it's like 2-6 minutes long and hangs on Probe EDD step at the start of the boot. Just sits there along time, then boots normally and the machine is fast and responsive and fully usable.

Both normal and RAM2 mode are saving and working.

Also when it does finish booting the Dell's display is dimmed 50% and I need to use the key combo FN - up arrow to set the brightness to full again. Kind of odd.

The NTFS is working though and better than expected and a .ucimg management utility is going to be a real useful tool. Just some small weirdness and that long hang at the beginning of the boot cycle.

User avatar
wiak
Posts: 3665
Joined: Tue Dec 03, 2019 6:10 am
Location: Packing - big job
Has thanked: 56 times
Been thanked: 1027 times
Contact:

Re: Utility to Create, and Manage XXXXX.ucimg Disk partition Image Files

Post by wiak »

rockedge wrote: Tue Dec 13, 2022 2:55 pm

The NTFS is working though and better than expected and a .ucimg management utility is going to be a real useful tool. Just some small weirdness and that long hang at the beginning of the boot cycle.

That's certainly all a bit weird, but I haven't noticed the effect myself. Then again, I don't normally use ucimg savefile modes. I'll try sometime and obviously do my best to fix anything I come across, but if you come across any definite confirmable results please let me know. The fact it ends up working is the curious thing if having all these slowdown effects on your machine - I have no idea sorry.

https://www.tinylinux.info/
DOWNLOAD wd_multi for hundreds of 'distros' at your fingertips: viewtopic.php?p=99154#p99154
Αξίζει να μεταφραστεί;

User avatar
rockedge
Site Admin
Posts: 5774
Joined: Mon Dec 02, 2019 1:38 am
Location: Connecticut,U.S.A.
Has thanked: 2052 times
Been thanked: 2134 times
Contact:

Re: Utility to Create, and Manage XXXXX.ucimg Disk partition Image Files

Post by rockedge »

@wiak It hangs for a while on Probing EDD

IMG_20221213_154846_1.jpg
IMG_20221213_154846_1.jpg (22.18 KiB) Viewed 524 times

How does it get disabled? Should it be disabled?

Then continues to boot perfectly into RAM (normal) or RAM2 mode and will use successfully both the sparse image file and the fixed image file.

User avatar
wiak
Posts: 3665
Joined: Tue Dec 03, 2019 6:10 am
Location: Packing - big job
Has thanked: 56 times
Been thanked: 1027 times
Contact:

Re: Utility to Create, and Manage XXXXX.ucimg Disk partition Image Files

Post by wiak »

rockedge wrote: Tue Dec 13, 2022 8:58 pm

@wiak It hangs for a while on Probing EDD
IMG_20221213_154846_1.jpg

How does it get disabled? Should it be disabled?

Then continues to boot perfectly into RAM (normal) or RAM2 mode and will use successfully both the sparse image file and the fixed image file.

I know nothing about EDD, but I've seen this kind of thing before. Do you not just put edd=off on the kernel line or something? Actually I see that EDD off message a lot I think, but has no effect on my boots usually, but I've seem someone else saying all was fine after edd=off, but I can't rememeber who or what post.

By the way, talking about weird, I have found an old bionicpup32 iso on my system that claims to be under 12MB in size yet filemnt opens it and seems to have normal pup sfs files inside it... I have PM'd you a shared link of the iso I've put on owncloud since honestly my brain seems to be malfunctioning right now and I hope you can tell me what a junk file it is... I may even try booting it now just to confirm it is damaged... Maybe I have discovered a new iso compression method by chance that will shake the world of technology - em... I doubt this. EDIT: I have since tried copying files out of the mounted iso and no go... obviously just corrupt but header somehow allowing filemnt to apparently mount it... probably a part-download of some kind... too bad, thought I'd become famous.

Last edited by wiak on Tue Dec 13, 2022 9:43 pm, edited 3 times in total.

https://www.tinylinux.info/
DOWNLOAD wd_multi for hundreds of 'distros' at your fingertips: viewtopic.php?p=99154#p99154
Αξίζει να μεταφραστεί;

User avatar
wiak
Posts: 3665
Joined: Tue Dec 03, 2019 6:10 am
Location: Packing - big job
Has thanked: 56 times
Been thanked: 1027 times
Contact:

Re: Utility to Create, and Manage XXXXX.ucimg Disk partition Image Files

Post by wiak »

rockedge wrote: Tue Dec 13, 2022 8:58 pm

@wiak It hangs for a while on Probing EDD
IMG_20221213_154846_1.jpg

How does it get disabled? Should it be disabled?

Then continues to boot perfectly into RAM (normal) or RAM2 mode and will use successfully both the sparse image file and the fixed image file.

I remember some big hang in the past that also had that EDD message, but the hang was nothing to do with that - fredx181 reported it and I checked it very thoroughly only to determine it seemed to be a problem with a particular kernel version. The problem subsequently went away with different kernel releases. Again, I have no idea if the same sort of issue or where that long ago problem was seen or posted about. I do remember we were messing around with loading different modules in different orders since seemed to effect matters, but in the end it was just a limit to the modules number or something somehow that the kernel was wrongly forcing and only fix was different kernel. But what is happening on your machine may well be entirely different issue. Most I can do just now is try making ucimg file using you sparse and non-sparse methods and seeing if boots or not fine on my machine - if it does boot without issue then won't tell you much...

https://www.tinylinux.info/
DOWNLOAD wd_multi for hundreds of 'distros' at your fingertips: viewtopic.php?p=99154#p99154
Αξίζει να μεταφραστεί;

User avatar
wiak
Posts: 3665
Joined: Tue Dec 03, 2019 6:10 am
Location: Packing - big job
Has thanked: 56 times
Been thanked: 1027 times
Contact:

Re: Utility to Create, and Manage XXXXX.ucimg Disk partition Image Files

Post by wiak »

rockedge wrote: Tue Dec 13, 2022 8:58 pm

@wiak It hangs for a while on Probing EDD
IMG_20221213_154846_1.jpg

How does it get disabled? Should it be disabled?

Then continues to boot perfectly into RAM (normal) or RAM2 mode and will use successfully both the sparse image file and the fixed image file.

I made 1GB sparse upper_changes.ucimg per your command and installed that along with KLV to ntfs partition on my nvme internal drive. I didn't do anything about EDD. Just booted per wd_grubconfig suggested stanza into RAM2 mode. Booted fine without delay and had expected background with moon in it. So I can't confirm your findings on my machine sorry. All booted okay and persistence seems to be working fine.

EDIT: sorry, booted from ext4 partition. Mistake. Will move it all over to ntfs partition and see if still works, boots quick, and I'll report back in a moment...

Attachments
sparse_ucimg_on_ntfs.jpg
sparse_ucimg_on_ntfs.jpg (57.54 KiB) Viewed 488 times

https://www.tinylinux.info/
DOWNLOAD wd_multi for hundreds of 'distros' at your fingertips: viewtopic.php?p=99154#p99154
Αξίζει να μεταφραστεί;

User avatar
wiak
Posts: 3665
Joined: Tue Dec 03, 2019 6:10 am
Location: Packing - big job
Has thanked: 56 times
Been thanked: 1027 times
Contact:

Re: Utility to Create, and Manage XXXXX.ucimg Disk partition Image Files

Post by wiak »

Sorry rockedge, I can't do any ntfs testing at the moment. I've run into a problem with my laptop - something to do with its amazing secure boot, which I've now turned off, but still not letting me access the ntfs and acting weird more generally. Like I said, my brain not working today isn't helping either.

Maybe Soniya is installing to ntfs? If so could let you know if having any issues with upper_changes.ucimg files? Long time since I tried. Unfortunatly in haste I deleted my old test klv folder that was already working in ntfs partition via old version of klv, so I can't even backtrack. I don't think the initrd has been changed at all since adding ntfs support aside from that minor 'fuse' addition, and I certainly had no slow boot problem when I did test it back then.

https://www.tinylinux.info/
DOWNLOAD wd_multi for hundreds of 'distros' at your fingertips: viewtopic.php?p=99154#p99154
Αξίζει να μεταφραστεί;

User avatar
rockedge
Site Admin
Posts: 5774
Joined: Mon Dec 02, 2019 1:38 am
Location: Connecticut,U.S.A.
Has thanked: 2052 times
Been thanked: 2134 times
Contact:

Re: Utility to Create, and Manage XXXXX.ucimg Disk partition Image Files

Post by rockedge »

@wiak It could just be this particular machine. KLV-Airedale-rc2 boots fast from the internal HDD and from a 16 GiB usb stick drive formatted to ext4. It might be that this super cheap 8 gb usb stick formatted into 2 partitions. A small FAT32 for the bootloader and menu.lst and the rest is NTFS.
Once it is started it runs really well. It's fast and responsive and the save mechanism in RAM2 mode is working and also.

The only small thing is those 3-5 minutes it hangs at that probe edd step. The rest of the boot is pretty fast. I am going to test it running from a NTFS and a Windows 10 installation. I use it for tax software and the machine can already dual boot frugally installed Puppy and FirstRib's. While it was a Windows system I ran Grub4Dos on the HDD and then chained the Windows bootloader from the menu.lst entry and set up the Puppy and FB boot stanza's and it all worked. I read later none of what I did should work but here we are. About ready to test the .ucimg mechanism in a KLV frugal installation on this Dell INSPIRON 1505E laptop.

This will help to determine if a problem in fact does exist.

I'll have to research what the probing EDD is,does and how to control it.

Update: edd=Enhanced Disk Drive

adding edd=off to the end of the kernel command line did disable it. Surprised at much verbosity disappeared during the boot after it was disabled.

Still took 4-5 minutes to boot but once going is really okay

Code: Select all

title Linux KLV-Airedale-rc2 (usb)
  find --set-root uuid () 7309CA194510E31F
  kernel /KLV-Airedale-rc2/vmlinuz w_bootfrom=UUID=7309CA194510E31F=/KLV-Airedale-rc2 net.ifnames=0 edd=off
  initrd /KLV-Airedale-rc2/initrd.gz
User avatar
wiak
Posts: 3665
Joined: Tue Dec 03, 2019 6:10 am
Location: Packing - big job
Has thanked: 56 times
Been thanked: 1027 times
Contact:

Re: Utility to Create, and Manage XXXXX.ucimg Disk partition Image Files

Post by wiak »

rockedge wrote: Wed Dec 14, 2022 1:11 am

@wiak It could just be this particular machine. KLV-Airedale-rc2 boots fast from the internal HDD and from a 16 GiB usb stick drive formatted to ext4. It might be that this super cheap 8 gb usb stick formatted into 2 partitions. A small FAT32 for the bootloader and menu.lst and the rest is NTFS.
Once it is started it runs really well. It's fast and responsive and the save mechanism in RAM2 mode is working and also.

Well, my brain started working and I was able to test ntfs booting via upper_changes.ucimg savefile.

There is unfortunately a problem with KLV rc2, at least as far as my computer is concerned. It will boot and use a ucimg savefile if underlying partition is ext4, but not if it is ntfs... Made me suspect the kernel.

After a bit timeline-related detective work, I determined that the KLV version I successfully used booting using ucimg savefile was quite a while back using KLV beta19; I hadn't tried ntfs since. So I tried beta19 again, and sure enough, no problem at all booting using ucimg savefile on ntfs partition on my computer with new download/install of beta19. In a parallel frugal install on same drive I had rc2 (also a pristine new frugal install with fresh ucimg), which would not boot using identical empty ucimg savefile... So I temporarily removed rc2 versions of vmlinuz, initrd.gz (which I note is actually an xz), 00modules, and 01firmware and replaced these with the ones from beta19 and it then booted fine.

There has only been one change in skeleton initrd since beta19 and that is that simple addition of fuse module in one code line. I checked that was the only difference between the beta19 w_init and that of rc2 using diff and confirm they are otherwise identical. That leads me to the clear determination (I believe) that there is something about the kernel/modules/firmware combination used in rc2 that is not correctly mounting ntfs partition. The initrd relies on the kernel having the new ntfs3 support (since it uses mount -t ntfs3 as the code to mount the underlying ntfs partition where the ucimg is stored - that being for the case when partition format is ntfs of course. For partiton ext4 the initrd doesn't use mount -t ntfs of course... so ext4 mount of underlying frugal installs is working with ucimg in rc2 but ntfs underlying mount is not.

So, something wrong with the rc2 kernel/modules/firmware in my opinion. The kernel is above version 5.15 so ntfs3 support should be in there, but the other possible error might be fred's Thomas M utility for slimming modules that you added to the skeleton initrd - I have a feeling I mentioned something before about ntfs3 support related to that(?) - not sure, but check that too.

EDIT: Here is the post fred fixed his script to include ntfs3 support. Hope you aren't using older version of his slim modules script rockedge:
viewtopic.php?p=69909#p69909

fredx181 wrote: Thu Oct 20, 2022 5:56 pm

Update 2022-11-02, script replaced below, added ntfs3 to the kernel modules to be copied to the initrd, thanks @wiak see: viewtopic.php?p=71211#p71211

What I had found, and posted for fred's attention was:

wiak wrote: Wed Nov 02, 2022 7:30 am

NOTE WELL: On looking at newest version of Thomas's initramfs_create script I note that the version being used with KLV seems to have missed out:

Code: Select all

copy_including_deps /$LMK/kernel/fs/ntfs3

That is needed also in KLV builds since the FirstRib initrd-latest used ntfs3 module for savefiles being used with ntfs partitions.

I have one last diff check to make to ensure no other change to the skeleton initrd, which is to check the init itself (only did a diff of the beta19 and rc2 w_init files thus far, so still to double check no change in init itself, but I doubt there is any change there).

Since I don't normally use ntfs or savefiles I don't know which release last worked correctly with ntfs on my machine (but if you need to check back then start with that beta19 since successful with ntfs). So overall, I'm suspecting that ntfs3-related module missing in fred's slim utility version that you are using. An easy fix therefore!

https://www.tinylinux.info/
DOWNLOAD wd_multi for hundreds of 'distros' at your fingertips: viewtopic.php?p=99154#p99154
Αξίζει να μεταφραστεί;

User avatar
rockedge
Site Admin
Posts: 5774
Joined: Mon Dec 02, 2019 1:38 am
Location: Connecticut,U.S.A.
Has thanked: 2052 times
Been thanked: 2134 times
Contact:

Re: Utility to Create, and Manage XXXXX.ucimg Disk partition Image Files

Post by rockedge »

@wiak I think that is it! Wrong cr-debkernel script. Which one is it now? It is 5:45 in morning and I need a simple link to the correct script......

User avatar
wiak
Posts: 3665
Joined: Tue Dec 03, 2019 6:10 am
Location: Packing - big job
Has thanked: 56 times
Been thanked: 1027 times
Contact:

Re: Utility to Create, and Manage XXXXX.ucimg Disk partition Image Files

Post by wiak »

rockedge wrote: Wed Dec 14, 2022 10:54 am

@wiak I think that is it! Wrong cr-debkernel script. Which one is it now? It is 5:45 in morning and I need a simple link to the correct script......

It is the link I gave you: viewtopic.php?p=69909#p69909
The latest cr script is in that top post I believe. Just search to see if ntfs3 module mentioned in it and you have the right one.

https://www.tinylinux.info/
DOWNLOAD wd_multi for hundreds of 'distros' at your fingertips: viewtopic.php?p=99154#p99154
Αξίζει να μεταφραστεί;

User avatar
rockedge
Site Admin
Posts: 5774
Joined: Mon Dec 02, 2019 1:38 am
Location: Connecticut,U.S.A.
Has thanked: 2052 times
Been thanked: 2134 times
Contact:

Re: Utility to Create, and Manage XXXXX.ucimg Disk partition Image Files

Post by rockedge »

@wiak So I began to try out different combinations of kernel parts and I found out that when I just used the original skeleton initrd.gz it all booted in one smooth fast sequence!

I will be digging deeper but this combo boots a KLV-Airedale-rc2 on a NTFS partition on a USB thumb drive very quickly. Which is real relief! I thought I broke it.

This will need some more testing but the difference in boot speed between the skeleton initrd.gz and those from the Debian kernel builds is immense.

UPDATE: I tried a new kernel radky has built for FP96 and is being tested right now. So I changed the zdrv to a 00module for KLV by moving /lib to /usr/lib and replaced the components then booted on the old laptop. This is using just the skeleton initrd.gz!

And it boots, not as fast as the 5.16.14-KLV kernel I made awhile back does, but that one seemed to miss the complete soc support. The boot time is much much improved with a different kernel. I tried a brand new one just now built with Fred's script and that initrd.gz (xz) seems to be the cause.

User avatar
mikewalsh
Moderator
Posts: 5628
Joined: Tue Dec 03, 2019 1:40 pm
Location: King's Lynn, UK
Has thanked: 590 times
Been thanked: 1715 times

Re: Utility to Create, and Manage XXXXX.ucimg Disk partition Image Files

Post by mikewalsh »

@rockedge :-

Re; "Probing EDD..."

As stated earlier, literally all you do is to add

Code: Select all

"edd=off"

.....to the end of the kernel line. That's all there is to it.

The only times I've had to do this, recently, was when installing the last couple of ZorinOS releases. Also when experimenting with a 'frugal' Manjaro using Will's scripts, so.....I'm inclined to agree that it's something that's been added to recent kernel builds.

Having said which, I'm using the k6.1.0 kernel peebee built the other morning in my daily driver.......jrb's 'lite' spin on BK's old Quirky April 7.0.1 (which has also had an upgrade to Bionic's glibc 2.27). Touch wood, it all seems to be performing really very well with the upgrades (including Fossa's firmware SFS) - quite remarkable for an 8-yr old distro.....and no 'Probing EDD...' with k6.1.0.

It's turning into very much of a "Frankenpuppy", I admit.....but a very, very stable one, for all that.

Mike. ;)

Puppy "stuff" ~ MORE Puppy "stuff" ~ ....and MORE! :D
_______________________________________________________

Image

User avatar
wiak
Posts: 3665
Joined: Tue Dec 03, 2019 6:10 am
Location: Packing - big job
Has thanked: 56 times
Been thanked: 1027 times
Contact:

Re: Utility to Create, and Manage XXXXX.ucimg Disk partition Image Files

Post by wiak »

rockedge wrote: Wed Dec 14, 2022 1:34 pm

...built with Fred's script and that initrd.gz (xz) seems to be the cause.

I'm looking into it further this morning here.

It also dawned on me that you could be using Fred's newest script and a module for ntfs3 is included, but I need to check I've included code to load that module in the initrd/init. Can't remember if I've done that - when using skeleton initrd you probably have ntfs3 built directly into the kernel since huge kernel situation. I quite possibly therefore haven't needed ntfs3 in the modprobe kernel loop in initrd/init until recently so need to do that now. I'll report back after checking if already there or not.

EDIT: quick look at modules.dep and modules.builtin suggest ntfs3 is not built into the kernel so I presume that means module is required, but modules.dep doesn't show it as included as a module, so I'm guessing you used old version of fred's script, but wouldn't have worked anyway since I haven't included ntfs3 as a module (thinking previously that it would always be built into the kernel, which is of course not always true at all). I will manually see if I can get my rc2 klv working with ntfs today and then double-confirm that module needs included in the initrd and I will then upload also new skeleton initrd that contains that extra modprobe ntfs3 entry in the loop code.

EDIT2: I've also checked beta19 version, and ntfs3 is indeed built directly into the kernel with that one so no ntfs3 module was required (I checked modules.builtin which was in 00modules sfs). Two mistakes have therefore been made in the rc2 release related to this matter:
One that old fred cr initrd script was used druing the build that did not include ntfs3 module inside the initrd/usr/lib/modules, so solution there is to use latest version of that cr init script during the build as I indicated earlier.
Two that I wrongly imagined ntfs3 would always be provided directly builtin to the kernel, so I need to amend skeleton initrd to include ntfs3 in modprobe loop code, which I'll do today. No need to go back to huge kernel method probably - just need these two fixes implemented and hopefully ntfs3 ucimg support will be fine thereafter.

EDIT3: Seems KLV rc2 doesn't include ntfs3 at all... neither as a kernel inbuilt nor as a module. My comment assumes that I should see it mentioned in either modules.builtin or in modules.dep, but it is not in either so I can't fix to boot on my system if frugal installed to ntfs partition. So main issue seems to be no ntfs3 support provided in KLV RC2 at all. i.e. any kernel compile needs CONFIG_NTFS3_FS set appropriately to provide either module or builtin ntfs3 support.

Last edited by wiak on Wed Dec 14, 2022 10:24 pm, edited 3 times in total.

https://www.tinylinux.info/
DOWNLOAD wd_multi for hundreds of 'distros' at your fingertips: viewtopic.php?p=99154#p99154
Αξίζει να μεταφραστεί;

User avatar
wiak
Posts: 3665
Joined: Tue Dec 03, 2019 6:10 am
Location: Packing - big job
Has thanked: 56 times
Been thanked: 1027 times
Contact:

Re: Utility to Create, and Manage XXXXX.ucimg Disk partition Image Files

Post by wiak »

By the way, the OS I am installing from, Zorin lite 16.2, uses kernel 5.15.0, which was the first to come with ntfs3 support. However, that was a bit flaky and fixes for ntfs3 were put into 5.19. Indeed, I have seen some flaky ntfs3 behaviour when reading and writing to my windows ntfs partition from zorin to test klv there - I ended up losing 'transport end point' this morning and had to boot into windows proper which automatically noticed ntfs disk error, scanned and fixed it, and I had to delete my KLV-related folder using windows itself to clean up the issue. Hopefully using kernel 5.19 and above will have fixed the flakiness such that problems like that do not surface...

Come to think of it, I haven't checked if zorin is mounting my windows partition via ntfs3 or via ntfs-3g. Whichever it is using has turned out imperfect - to be fair it was fine right up until this morning... I've been reading and writing to the ntfs partition big-time so eventually something broke...

https://www.tinylinux.info/
DOWNLOAD wd_multi for hundreds of 'distros' at your fingertips: viewtopic.php?p=99154#p99154
Αξίζει να μεταφραστεί;

User avatar
wiak
Posts: 3665
Joined: Tue Dec 03, 2019 6:10 am
Location: Packing - big job
Has thanked: 56 times
Been thanked: 1027 times
Contact:

Re: Utility to Create, and Manage XXXXX.ucimg Disk partition Image Files

Post by wiak »

skel initrd version="6.0.0"; revision="-rc3" Date: 15 Dec 20

@rockedge I have now uploaded new versions of FR skeleton initrd-latest.gz and w_init-latest to owncloud.

The usual get initrd-latest script should automatically fetch these latest versions

Only change is the inclusion of ntfs3 in initrd/init modprobe for-loop code for possible case of ntfs3 only being supported by kernel as a module.

Any kernel compile needs CONFIG_NTFS3_FS set appropriately to provide either module or builtin ntfs3 support

get skeleton initrd script always here:
https://firstrib.rockedge.org/viewtopic ... 21944#p355

https://www.tinylinux.info/
DOWNLOAD wd_multi for hundreds of 'distros' at your fingertips: viewtopic.php?p=99154#p99154
Αξίζει να μεταφραστεί;

User avatar
rockedge
Site Admin
Posts: 5774
Joined: Mon Dec 02, 2019 1:38 am
Location: Connecticut,U.S.A.
Has thanked: 2052 times
Been thanked: 2134 times
Contact:

Re: Utility to Create, and Manage XXXXX.ucimg Disk partition Image Files

Post by rockedge »

@wiak I've downloaded the latest initrd.gz and am using it on a NTFS usb stick with rc2. The kernel is 6.0.12 that I just compiled.
This boots fast and smooth.

I tried again the latest cr-debkernel script Debian derivative build which is using the initrd-latest but the kernel panicked.

The Debian kernel is working great on any system other than NTFS and FAT32.

My compiled kernel is booting the NTFS usb drive with KLV-rc2 and just the raw skeleton initrd.gz and this has been smooth as glass.
But my kernels are missing something somewhere so it's not working 100% with my blade server's MATOX graphics and seems to be lacking some audio SOC support.

Other than that the kernel-kit 6.0.12 I made is working great in rc2 on the old Dell Vostro 1500.

User avatar
wiak
Posts: 3665
Joined: Tue Dec 03, 2019 6:10 am
Location: Packing - big job
Has thanked: 56 times
Been thanked: 1027 times
Contact:

Re: Utility to Create, and Manage XXXXX.ucimg Disk partition Image Files

Post by wiak »

rockedge wrote: Thu Dec 15, 2022 12:59 am

But my kernels are missing something somewhere so it's not working 100% with my blade server's MATOX graphics and seems to be lacking some audio SOC support.

Other than that the kernel-kit 6.0.12 I made is working great in rc2 on the old Dell Vostro 1500.

What I'd do is use build_firstrib_rootfs, latest version as provided in KLA iso, to make a small Void Linux based root filesystem but including official Void kernel (xbps-install -Sy linux6.0), modules and firmware in it, and then use Tomas M Slax method to slim down the initrd via the latest two utility scripts I also provided in KLA iso FRmake_initrd.sh plus its Thomas M code dependency I named FRmake_initrd_dep.sh. It's just a matter of running the FRmake_initrd.sh script after the build rootfs, though you'd manually have to make a 00modulesXXX.sfs and 01firmwareXXX.sfs out of the resulting firstrib_rootfs. Then you'd build your main Void-based KLV-Airedale without kernel firmware and use the specially made initrd.img along with the 00modules and 01firmware and related vmlinuz component. Yes, I am moving towards using name initrd.img since can then be gz or xz or zst form without having to change extension name all the time. The make_grubconfig script in KLA iso also prints out grub code in initrd.img form. To save re-reading all above in KLA second post, here is the relevant extract though you'd build a minimum void distro per usual rather than arch linux base. I'm hoping the resultant official Void kernel and slimmed modules initrd.img might contain all of ntfs3 and the soc stuff:

wiak wrote: Thu Nov 24, 2022 5:09 am

REBUILDING FROM SCRATCH

If you ever want to rebuild the 07firstrib_rootfs (the main distro root filesystem) the way to do so is simplicity itself:

1. Open a terminal at the directory where you have your KLA_OT2 frugal install.

2. Enter the command:

Code: Select all

./build_firstrib_rootfs.sh arch default amd64 f_00_Arch_amd64-openboxFull_jgmenu_600r2.plug

(ignore any trap and tar error messages...)

Of course you could first edit the text plugin file f_00_Arch_amd64-openboxFull_jgmenu_600r2.plug to add or delete whatever apps and so on you wish first... simple build system you see.

Sit back and drink some coffee. A new directory called firstrib_rootfs containing the new root filesystem will be automatically created.

3. You can now disable the previous 07firstrib_rootfs.sfs by putting a D in front of the filename (i.e. D07firstrib_rootfs.sfs).

4. Assuming you continue to use the vupup kernel you don't need to modify the initrd.img so all you need to do to run your new build is change name of directory firstrib_rootfs to 07firstrib_rootfs. Note that you do not need to squash that up to an sfs (unless you want to) - it will be used as is.

Now just reboot and new build should be working.

TOTALLY OPTIONAL: If you instead wanted to build a new initrd.img (for those who are planning to use official Arch kernel):

1. Once directory firstrib_rootfs build has completed (prior to renaming the directory):

run the command:

Code: Select all

./FRmake_initrd.sh latest

A new initrdXXX.img file will be auto created and you can rename it to initrd.img so it works with your already configured grub.

I'll certainly try the above at home and if you haven't by the time I finish I'll let you know how it went. (I'll probably just chroot into your existing KLV RC2 root filesystem using mount_chroot.sh and add Void official kernel/modules to that then exit the chroot and run ./FRmake_initrd.sh on the result to create a slim initrd.img followed by making 00modules sfs after extracting into own directory out of KLV root filesystem if you see what I mean. I'll just use that Debian 01firmware, or wherever it is from..., as first try anyway.

Actually, you could just go ahead and build_firstrib_rootfs with that new script your actual KLV root filesystem, then run the FRmake_initrd.sh script (it has a simple --help option) and then just manually take the firmware and modules out of the firstrib_rootfs and make them into 01firmware and 00modules sfs files, or write a simple little extra script to do that last part including extracting vmlinuz and clearing any files/dirs out of the rootfs you don't want. FRmake_initrd automatically also fetched initrd-latest prior to auto-adding slimmed modules to it. EDIT: However, Void provides too much firmware, so come to think of it you should use 01firmware from similar kernel you've been providing.

https://www.tinylinux.info/
DOWNLOAD wd_multi for hundreds of 'distros' at your fingertips: viewtopic.php?p=99154#p99154
Αξίζει να μεταφραστεί;

User avatar
wiak
Posts: 3665
Joined: Tue Dec 03, 2019 6:10 am
Location: Packing - big job
Has thanked: 56 times
Been thanked: 1027 times
Contact:

Re: Utility to Create, and Manage XXXXX.ucimg Disk partition Image Files

Post by wiak »

Above idea, using Void official 6.0.12 kernel with FRmake_initrd.sh using Thomas M method to slim down appropriate initrd.img and then simply extracting modules and vmlinux out to make 00modules and using with your previous 01firmware sfs seems to have worked perfectly and on first attempt... I note the Void modules are individually gz compressed so can improve upon final 00modules sfs if I first uncompress the individual modules before making the sfs I think though. I will write a extra script to be run after FRmake is finished to do that modules uncompression (followed by appropriate depmod rebuild) and then extract the modules and vmlinuz appropriately and also give you a copy of the kernel/modules/initrd.img result that is working for me. It works including the sof support for my tricky machine - just about to try the ntfs3 boot (which I admit to not having tried with this one yet; here is hoping...). So this method and both the FRmake_initrd.sh and FRextract_kernel.sh method will also work for KLA builds (which also removes the issue that Arch Linux compress modules with zst that busybox cannot modprobe).

So this new FirstRib-style build mechanism, becomes that of running three scripts, one after the other:
1. build_firstrib_rootfs.sh ... f_plug... to include required official kernel/modules (but for booting, to save space, can use 01firmwareXXX.sfs from elsewhere is good enough match so don't need to include official Void firmware in build). Or use unsquashed existing 07KLV-airdaleXXX.sfs and "./mount_chroot.sh squashfs-root" to bring down new kernel/modules prior to steps 2 and 3 below.
The modules end up getting extracted and taken out of the main root_filesystem, by step 3 script below, prior to optionally making these extracted full set of modules into an sfs using mksquashfs
2. FRmake_initrd.sh latest <root_filesystem_name> <compression_type xz|zst|gz as default>, which fetches skeleton initrd-latest.gz and from that creates new initrd.img containing slimmed modules.
3. FRextract_kernel.sh <root_filesystem_name> followed by a final mksquashfs of the resulting extracted uncompressed modules directory

As explained in PM you can alternatively replace step 1 above by using an existing 07KLV-airedale_rootfs.sfs and manipulating it prior to steps 2 and 3.

For special pseudo-full-install case, the above arrangement should make it trivially easy to do full updates including of glibc and kernel/modules I think.

Of course the original two build script method using build_firstrib_rootfs and build_wiak_initrd will also be supported, though still working on some details for build_wiak_initrd. However, slimmed initrd, being optional, logically remains an optional extra build script step no matter how the final build gets glued together ready for booting. More than one way to skin the cat. In a sense FirstRib build design is like traditional UNIX in philosophy - simple tools chained together for different build effects rather than one monolithic whole though of course a glue script can be used to run whatever other parts are wanted one after the other so single script build effectively easily achievable. And no huge github resource required to pull in massive core parts - most everything required is directly provided via the simple scripts and plugins via reliable package manager, so a tiny build system to download and use immediately. Easy to develop, maintain and modify - so simple doesn't even need git to maintain it - so dev work via simple forum discussion works fine for it since just a few simple scripts to alter as and when useful. Yet it remains unique owing to its general purpose flexible overlayfs based initrd, using sfs or uncompressed filesystem layers, and easy to build into whatever shape and form wanted, via its simple build plugin system.

EDIT:
Did indeed all work. Booted right now using ucimg savefile from my windows 10 ntfs partition and sound sof audio and everything all working for me. I don't know about support for your graphics set up @rockedge but still using same 01firmware as you supplied for original KLV RC2. As I said, using official void linux latest 6.0.12 kernel/modules with the vmlinux and modules extracted from the resultant root filesystem (of course in the end I'm using original KLV RC2 root filesystem since didn't contain the modules there anyway) and made a 00modules.sfs; still to make that modules sfs a bit smaller via changed compression methods but enough to test all working smooth as silk. The FRmake_initrd.sh did it's business and result is initrd.img of size 9.2MiB (it is actually just gz compressed). I haven't yet tried sparse upper_changes.ucimg, just using my old empty 64MB one in tests. I'll try your sparse ucimg in next set of tests though. Also working fine with w_changes=RAM2 mode per attached screenshot.

EDIT2: your sparse file upper_changes.ucimg also working on the ntfs partition including RAM2 mode when using this official Void kernel with slimmed initrd.img and 00modules.sfs (and original 07KLV-airedale_rootfs.sfs)

Attachments
KLV_RC2_VoidKernel_ntfs_ucimg.jpg
KLV_RC2_VoidKernel_ntfs_ucimg.jpg (66.73 KiB) Viewed 492 times

https://www.tinylinux.info/
DOWNLOAD wd_multi for hundreds of 'distros' at your fingertips: viewtopic.php?p=99154#p99154
Αξίζει να μεταφραστεί;

User avatar
rockedge
Site Admin
Posts: 5774
Joined: Mon Dec 02, 2019 1:38 am
Location: Connecticut,U.S.A.
Has thanked: 2052 times
Been thanked: 2134 times
Contact:

Re: Utility to Create, and Manage XXXXX.ucimg Disk partition Image Files

Post by rockedge »

@wiak With the scripts and instructions I've now included a Void Linux kernel into KLV! The new kernel combination is with the firmware from a huge Puppy Linux kernel 6.0.0 and works perfectly on FAT32, NTFS, ext2/3/4 formatted partitions.

The boot times starting from a NTFS partition is surprisingly fast and is smooth even on a DELL Vostro 1500 laptop that has a lot of miles under it's feet. It was a Windows XP laptop for a pre-school classroom of 4-6 year old kids for more than 12 years. Speaks for itself.

I am pleased with the preliminary results of operations testing of KLV-Airedale-rc3 equipped with a Void Linux 6.0.12_1 kernel, so it will be uploaded to the usual location.

Post Reply

Return to “KL-Dev_Work”