Booting Puppy ISO files, enhanced loopback.cfg, utility to do it

Under development: PCMCIA, wireless, etc.

Moderator: Forum moderators

gyrog
Posts: 594
Joined: Thu Oct 01, 2020 8:17 am
Location: Australia
Has thanked: 14 times
Been thanked: 180 times
Contact:

Booting Puppy ISO files, enhanced loopback.cfg, utility to do it

Post by gyrog »

Here are a couple of CLI utilities to enhance booting Puppy ISO files as a grub2 loopback device.

1) mk-grub2-cfgs generates an enhanced 'loopback.cfg' and a slightly enhanced 'grub.cfg',
as alternatives to the ones already in a release Puppy ISO file.

This utility takes a single parameter, the $NAME of the Puppy to use in the grub2 config files.
If no parameter is given, "Puppy Linux" is used as the $NAME.
The enhancements to 'loopback.cg' are:
support for an optional second passed grub variable, $rootuuid.
support for optional presence of 'local-initrd.gz'.
displays a message on the console indicating which files are being loaded by the 'initrd' line.
and adds 'pfix=fsckp'.

The $rootuuid grub variable contains the UUID of the partition where the ISO file resides.
If a $rootuuid variable is found, it is used, else the config file attempts to generate it.
Any $rootuuid variable is passed on to Puppy's 'isoboot' script as an "img_dev=" boot parameter.
This means that the 'isoboot' script does not have to wait 5 seconds in case of a slow device,
and it does not have to search each partition looking for the specified ISO file.

The $rootuuid is processed this way because booting utilities frequently do not provide it,
and Puppy will most likely still boot without it, just not as efficiently.
I found this variable name being used in the Manjaro grub2 stuff. (I did not just make it up.)

Run mk-grub2-cfgs.
Use an ISO editor like "ISOMaster" to replace the /boot/grub/loopback.cfg with the new loopback.cfg.

The current woof-ce initrd.gz supports this, so there is no need for a special Puppy download.

Since the grub2 installed on your computer is booting Puppy using the files contined in the ISO file,
it does not matter if the edited ISO file is just a "data" ISO.
Being able to easily add files to, and remove files from, the ISO is an advantage of this approach.

This utility should disappear if/when the appropriate changes are made to '/woof-code/support/mk_iso.sh'.
It has been created by extracting the relevant parts of 'mk_iso.sh' and changing them.
So it should be no problem to generate a woof-ce patch.

2) mk-img-boot-entries creates boot entries for the grub2 in your computer
so it can boot using the contents of a specified ISO file utilizing it's /boot/grub/loopback.cfg.

It accepts 2 parameters with the form <partition>:<relative path>.

The first parameter specifies the location of the ISO files to be processed.
Each image file in this location is processed,
if it contains a /boot/grub/loopback.cfg, a boot entry is generated, otherwise it is ignored.

The second paramater specfies the grub file to contain the generated boot entries,
this file is replaced by each run of the utility, and must be on the partition where grub2 is installed.

If the config file $HOME/.config/img-boot-entries.conf exists, this will be used for the parameters
if none are supplied on the command line.

If the utility creates the file /images.cfg, then /grub.cfg needs to be edited to include:

Code: Select all

if [ -e /images.cfg ]; then
  source /images.cfg
fi

A separate file is used, so it can be simply replaced by each run of the utility.

The concept is, if you add or delete an image file to the specified location,
then run mk-img-boot-entries, when you reboot, the image files available for booting via the grub2 menu has been updated.

This utility works with current Puppy release ISO files, it does not require mk-grub2-cfgs,
but without the patched /boot/grub/loopback.cfg, the boot will be slower.
It might also work with any ISO file that contains a /boot/grub/loopback.cfg,
since the generated boot entries do not contain anything peculiar to Puppy.

How does it all work?
It starts with Puppy ISO files containing a "rich" /boot/grub/loopback.cfg.
Hence mk-grub2-cfgs.

Your computer boots a partition containing grub2.
There are entries in the menu to boot an image file as a loopback device.
These entries could be generated by mk-img-boot-entries.
The selected entry should do the following:
set and make avaiiable a $rootuuid variable for the UUID of the patition containing the ISO file
set and make available an $iso_path variable for the relative path/filename of the ISO file
read /boot/grub/loopback.cfg from the ISO file, and activate it's grub menu
read vmlinuz, ucode.cpio, initrd.gz, local-initrd.gz, from the ISO file, and load them using the parameters defined by the selected boot entry in /boot/grub/loopback.cfg.

If you boot a Puppy ISO file, the 'isoboot' script locates the ISO file and mounts it as a loop device.
The 'init' script continues the boot using the files in the ISO file.

Note1: The grub within the ISO file is never used by this process, it can be safely removed from the ISO file.
If removing grub files from the ISO, make sure you leave:
/boot/grub/loopback.cfg
/boot/grub/font.pf2 (used by loopback.cfg)

Note2: Ventoy does not use this process, so does not trigger 'isoboot'.
Booting currrent Puppy Linux ISO files with current Ventoy, produces grief. Don't bother.

Files

mk-grub2-cfgs.gz
gunzip and place in $HOME/my-applications/bin
(1.2 KiB) Downloaded 26 times
img-boot-entries.conf.gz
gunzip and place in $HOME/.config
(642 Bytes) Downloaded 25 times

Edit: I have removed the 'mk-img-boot-entries.gz' because it has a bug. For replacement see viewtopic.php?p=103718#p103718

Last edited by gyrog on Fri Nov 17, 2023 10:32 am, edited 2 times in total.
User avatar
bigpup
Moderator
Posts: 6268
Joined: Tue Jul 14, 2020 11:19 pm
Location: Earth, South Eastern U.S.
Has thanked: 732 times
Been thanked: 1292 times

Re: Booting Puppy ISO files, enhanced loopback.cfg, utility to do it

Post by bigpup »

You give info on how this is suppose to work.

If I wanted to place a Puppy ISO on a USB stick.

How about a step by step info on what exactly do I do?

Forum Global Moderator
The things you do not tell us, are usually the clue to fixing the problem.
When I was a kid, I wanted to be older.
This is not what I expected :o

gyrog
Posts: 594
Joined: Thu Oct 01, 2020 8:17 am
Location: Australia
Has thanked: 14 times
Been thanked: 180 times
Contact:

Re: Booting Puppy ISO files, enhanced loopback.cfg, utility to do it

Post by gyrog »

bigpup wrote: Wed Nov 15, 2023 2:53 pm

If I wanted to place a Puppy ISO on a USB stick.

How about a step by step info on what exactly do I do?

1) Setup a bootable usb stick, with grub2 installed.
(This could be done with StickPup, and then delete the installed Puppy.)

2) For covenience, set a Label on the usb stick partition.
e.g run "fatlabel /dev/sdd1 FRED"

3) Make a directory on the usb stick called "isos", (or whatever).

4) Edit /grub.cfg and replace the boot entry that use to boot the now deleted Puppy to:

Code: Select all

if [ -e /images.cfg ]; then
  source /images.cfg
fi

5) Copy the ISO file to the usb stick in the /isos directory.

6) run "mk-img-boot-entries FRED:/isos FRED:/images.cfg"

7) Boot the usb stick and select the ISO file from the grub2 menu.

Steps 5) thru 7) can be repeated to boot more ISO files.

Note: If you edit $HOME/.config/img-boot-entries.conf with the "FRED:/isos" and "FRED:/images.cfg" parameters,
step 6) becomes, run "mk-img-boot-entries".

Edit:
If you want persistence for the Puppy ISO files in "FRED:/isos", create a SAVESPEC file in the "FRED:/isos" directory.

Clarity
Posts: 3270
Joined: Fri Jul 24, 2020 10:59 pm
Has thanked: 1347 times
Been thanked: 438 times

Re: Booting Puppy ISO files, enhanced loopback.cfg, utility to do it

Post by Clarity »

Hello @gyrog and THANKS for your efforts and guidance in this thread.

gyrog wrote: Wed Nov 15, 2023 3:40 pm

...
Edit:
If you want persistence for the Puppy ISO files in "FRED:/isos", create a SAVESPEC file in the "FRED:/isos" directory.

SAVESPEC file
I am assuming it remains the same as it has over the past years: Namely Drive label and folder of persistence files are maintained.

  • Questions
    Since no blanks are used in both label or persistence folder name, does it need to be contained within quotes?

  • Next

    • Could your "SAVESPEC generator utility" be submitted to WoofCE development to be added in future PUPs?

NOTE to users

  • SAVESPEC presence negates the need for any adjustments to a boot stanza for persistence's location files to be found and used, automatically, by the booting WoofCE PUP. It works no matter if you are booting pristine or if you have sessions saved from before.

  • Advice: Keep ALL of your WoofCE distro sessions in the same folder for all of your PUPs. Puppy is smart enough to detect, setup, and use that folder for its persistence management for you. This will eliminate many/most/all of the past hardship many have run into in the past with persistence;

    • on boots

    • as well as prior to reboots.

gyrog
Posts: 594
Joined: Thu Oct 01, 2020 8:17 am
Location: Australia
Has thanked: 14 times
Been thanked: 180 times
Contact:

Re: Booting Puppy ISO files, enhanced loopback.cfg, utility to do it

Post by gyrog »

There is a bug in the previous version of the 'mk-img-entries' script.
If you choose to store you ISO files in a directory other than '/boot-isos',
It processes the ISO files from the specified directory, but the directory in the generated boot entry is always '/boot-isos'.
The attached 'mk-img-entries.gz' file, fixes the bug.

mk-img-boot-entries.gz
gunzip to produce 'mk-img-entries' script
(1.67 KiB) Downloaded 21 times
gyrog
Posts: 594
Joined: Thu Oct 01, 2020 8:17 am
Location: Australia
Has thanked: 14 times
Been thanked: 180 times
Contact:

Re: Booting Puppy ISO files, enhanced loopback.cfg, utility to do it

Post by gyrog »

Clarity wrote: Fri Nov 17, 2023 1:32 am

Questions
Since no blanks are used in both label or persistence folder name, does it need to be contained within quotes?

No. I tend to over use quotes just in case a filename contains a space character.
It's just being sure that there are only 2 parameters, no matter what the user substiutes for the provided example.

Clarity wrote: Fri Nov 17, 2023 1:32 am

Could your "SAVESPEC generator utility" be submitted to WoofCE development to be added in future PUPs?

It's not my call, I publish utilities in the Forum.
If users find them to be useful, that's fine.
If Puppy creators, consider them worthy of acceptance into their Puppy, thats fine.
If I'm the only one who still uses them a little further down the track, thats fine.

I write stuff because I get an idea of how things might be done a little differently from the current standard Puppy.
This produces an itch to try and implement that idea.
If my proof of concept implementation seems to work, I publish it in the Forum.
What happens after that, depends on you folk.

gyrog
Posts: 594
Joined: Thu Oct 01, 2020 8:17 am
Location: Australia
Has thanked: 14 times
Been thanked: 180 times
Contact:

Re: Booting Puppy ISO files, enhanced loopback.cfg, utility to do it

Post by gyrog »

There is a downside to the process of booting ISO files via 'loopback.cfg',
it uses the grub2 that is installed in your computer, not the grub2 in the ISO file.
This may become a problem if the grub2 in your computer is a bit long in the tooth, and the 'loopback.cfg' depends on a recent grub2.

If grub2 was installed by FrugalPup, then it's a bit long in the tooth, it identifies as 2.03.
The current grub2 identifies as 2.06.

I have uploaded a 'grubx64.efi.gz' file to https://www.mediafire.com/folder/qiwvlxw46bp5f/iso, gunzip it to produce 'grubx64.efi'.
This identifies as "GNU GRUB version 2.06-13+deb12u1".
It's "borrowed" from a debian package.
It's unsigned, so it won't work with "Secure Boot" enabled.

To use:
In your efi fat32 boot partition, where grub2 is installed:
rename '/EFI/boot/grubx64.efi' file to something else, as a precaution.
Place the downloaded 'grubx64.efi' file in the '/EFI/boot' directory.

WARNING: Although it works fine for me, it is possible that it may not work in your computer.
So make sure you have an alternative means of booting your computer, before you try it.
(Or backup the partition as a clone, and make sure you have a working restore mechanism.)

gyrog
Posts: 594
Joined: Thu Oct 01, 2020 8:17 am
Location: Australia
Has thanked: 14 times
Been thanked: 180 times
Contact:

Re: Booting Puppy ISO files, enhanced loopback.cfg, utility to do it

Post by gyrog »

gyrog wrote: Wed Nov 15, 2023 11:02 am

Run mk-grub2-cfgs.
Use an ISO editor like "ISOMaster" to replace the /boot/grub/loopback.cfg with the new loopback.cfg.

The current woof-ce initrd.gz supports this, so there is no need for a special Puppy download.

The loopback.cfg generated by mk-grub2-cfgs depends on a patch to the isoboot script committed to woof-ce on Jun 18, 2021.
If the Puppy ISO file was not woof-ce'd after that date, the new loopback.cfg will not improve the efficiency of the boot.
This information should be available in the /etc/DISTRO_SPECS file, in the "BUILD_FROM_WOOF=" line.

gyrog
Posts: 594
Joined: Thu Oct 01, 2020 8:17 am
Location: Australia
Has thanked: 14 times
Been thanked: 180 times
Contact:

Re: Booting Puppy ISO files, enhanced loopback.cfg, utility to do it

Post by gyrog »

If you are booting a Puppy ISO that's a bit old and continues to issue a "Waiting 5 seconds" message,
you can add the attached 'local-initrd.gz' to the ISO file.
It contains patched 'init' and 'isoboot' scripts.

local-initrd.gz
(17.11 KiB) Downloaded 19 times
gyrog
Posts: 594
Joined: Thu Oct 01, 2020 8:17 am
Location: Australia
Has thanked: 14 times
Been thanked: 180 times
Contact:

Re: Booting Puppy ISO files, enhanced loopback.cfg, utility to do it

Post by gyrog »

Booting the release 'F96-CE_4.iso' fails.
After replacing the 'loopback.cfg' in the ISO with one generated by mk-grub2-cfgs, it boots fine.
Both boots using a grub2 boot entry generated by mk-grub2-cfgs.

User avatar
rockedge
Site Admin
Posts: 5711
Joined: Mon Dec 02, 2019 1:38 am
Location: Connecticut,U.S.A.
Has thanked: 1991 times
Been thanked: 2097 times
Contact:

Re: Booting Puppy ISO files, enhanced loopback.cfg, utility to do it

Post by rockedge »

@gyrog considering now whether or not to modify the F96-CE_4 ISO by adding in this capability and generating a F96-CE_5 with perhaps some other updates and tweaks. Wasn't in the original plan but might be worth the efforts to include some more updates and improve the ISO boot functionality.

User avatar
bigpup
Moderator
Posts: 6268
Joined: Tue Jul 14, 2020 11:19 pm
Location: Earth, South Eastern U.S.
Has thanked: 732 times
Been thanked: 1292 times

Re: Booting Puppy ISO files, enhanced loopback.cfg, utility to do it

Post by bigpup »

gyrog wrote: Sun Nov 19, 2023 6:51 am

If you are booting a Puppy ISO that's a bit old and continues to issue a "Waiting 5 seconds" message,
you can add the attached 'local-initrd.gz' to the ISO file.
It contains patched 'init' and 'isoboot' scripts.

Unless something has changed about older computers booting.

That 5 seconds delay was for the older computers, to have time to actually get USB devices fully up and active.
On some computers, the USB drive was not fully up and active, before the boot process tried to read from the USB drive Puppy was on.
Waiting 5seconds was the easy way to fix the issue.

Not an issue on newer or newest computers, where everything just works faster.

Forum Global Moderator
The things you do not tell us, are usually the clue to fixing the problem.
When I was a kid, I wanted to be older.
This is not what I expected :o

User avatar
rockedge
Site Admin
Posts: 5711
Joined: Mon Dec 02, 2019 1:38 am
Location: Connecticut,U.S.A.
Has thanked: 1991 times
Been thanked: 2097 times
Contact:

Re: Booting Puppy ISO files, enhanced loopback.cfg, utility to do it

Post by rockedge »

On some computers, the USB drive was not fully up and active, before the boot process tried to read from the USB drive Puppy was on.
Waiting 5seconds was the easy way to fix the issue.

Same in Kennel Linux variants...there is a kernel command line parameter to set a delay to wait for slow USB drives that @wiak built in the initrd.gz. This feature now is automatic and the boot delay can occur up to 30 seconds, but on most newer machines this isn't needed.

gyrog
Posts: 594
Joined: Thu Oct 01, 2020 8:17 am
Location: Australia
Has thanked: 14 times
Been thanked: 180 times
Contact:

Re: Booting Puppy ISO files, enhanced loopback.cfg, utility to do it

Post by gyrog »

bigpup wrote: Sun Nov 19, 2023 3:47 pm

That 5 seconds delay was for the older computers, to have time to actually get USB devices fully up and active.
On some computers, the USB drive was not fully up and active, before the boot process tried to read from the USB drive Puppy was on.
Waiting 5seconds was the easy way to fix the issue.

If the required device is known, then any waiting can be until that particicular device is known to 'blkid', so there is no uncessary waiting.
If the particular required device is not known, then there is not much choice other than to wait for some arbitary time just in case the required device happens to be a slow one, before the search of partitions is started.

This code provides the 'isoboot' script with the UUID of the partition that contains the ISO file, so it can wait just for that particular partition.
If the partition is fast, there will be no waiting, if the partition is slow, there will be waiting.
Also 'isoboot' does not have to do any searching, it can mount only the required partition, and find the ISO file there.

gyrog
Posts: 594
Joined: Thu Oct 01, 2020 8:17 am
Location: Australia
Has thanked: 14 times
Been thanked: 180 times
Contact:

Re: Booting Puppy ISO files, enhanced loopback.cfg, utility to do it

Post by gyrog »

@rockedge,
Since it can be done, there are some folk who will. It's not a bad idea to make the experience as good as reasonably possible.

I've tended to think that directly booting ISO files is only useful for quick testing of a new Puppy.
But having "exposed" the full grub2 menu that may be contained in the ISO, I am changing my mind, a bit.
I'm impressed at how easy it is to boot an ISO and select "pfix=rdsh", or "pfix=ram".
In my environment, these things involve editing my large 'grub.cfg', and when I'm done, to edit it back again.

It's got me thinking about doing something similar for frugal installs.
i.e. add a 'frugal.cfg' to the frugal install, and then when you select a particular Puppy in the main 'grub.cfg' menu,
you get this sub-menu from 'frugal.cfg', instead of simply booting the Puppy directly.
The 'frugal.cfg' could be generated by installers, (e.g. FrugalPup) based on the 'grub.cfg' in the ISO file.

I also avoided ISO files because they are pesky to change, once you "edit" them and burn them, the CD won't boot.
But booting the "edited" ISO will work.

Ideally there would be a utility that can generate a grub2 loopback ISO boot entry on the fly, for any selected ISO, and then boot it.
But a utility that statically generates the grub2 loopback ISO boot entries is all I can manage now. (mk-img-boot-entries).

User avatar
rockedge
Site Admin
Posts: 5711
Joined: Mon Dec 02, 2019 1:38 am
Location: Connecticut,U.S.A.
Has thanked: 1991 times
Been thanked: 2097 times
Contact:

Re: Booting Puppy ISO files, enhanced loopback.cfg, utility to do it

Post by rockedge »

This is the loopback.cfg that we use with a boot loader within the ISO, so the ISO can be burned as a CD-ROM or directly boot in virtual machines from ISO in QEMU and VirtualBox.

loopback.cfg

Code: Select all

set distname=KLV-Airedale
set revision=RT
set savepart=Persistence
set savedir=Sessions
set bootlabel=SG2D

loadfont /boot/grub/font.pf2

# https://help.ubuntu.com/community/Grub2/Displays
color_normal=cyan/black
#menu_color_highlight=black/light-gray
menu_color_highlight=yellow/red
menu_color_normal=light-gray/black

### loopback.cfg: All work with SG2D
menuentry "${distname}-${revision} (RAM0 no save persistence)" {
    linux /vmlinuz w_bootfrom=${iso_path} w_changes=RAM0 net.ifnames=0
    initrd /initrd.gz
}

menuentry "${distname}-${revision} (RAM2 save on demand to ${savepart}_${savedir})" {
    linux /vmlinuz w_bootfrom=${iso_path} w_changes=LABEL=${savepart}=/${savedir}/${distname}-${revision} w_changes1=RAM2 net.ifnames=0
    initrd /initrd.gz
}

menuentry "${distname}-${revision} (direct saves to ${savepart}_${savedir})" {
    linux /vmlinuz w_bootfrom=${iso_path} w_changes=LABEL=${savepart}=/${savedir}/${distname}-${revision} net.ifnames=0
    initrd /initrd.gz
}

menuentry "${distname}-${revision} (RAM2 save on demand to ${bootlabel}_${savedir})" {
    linux /vmlinuz w_bootfrom=${iso_path} w_changes=LABEL=${bootlabel}=/${savedir}/${distname}-${revision} w_changes1=RAM2 net.ifnames=0
    initrd /initrd.gz
}

menuentry "${distname}-${revision} (direct saves to ${bootlabel}_${savedir})" {
    linux /vmlinuz w_bootfrom=${iso_path} w_changes=LABEL=${bootlabel}=/${savedir}/${distname}-${revision} net.ifnames=0
    initrd /initrd.gz
}

menuentry "Shutdown" {
	halt
}

menuentry "Reboot" {
	reboot
}

grub.cfg

Code: Select all

set distname=KLV-Airedale
set revision=RT
set savepart=Persistence
set savedir=Sessions
set bootlabel=Ventoy

insmod part_acorn
insmod part_amiga
insmod part_apple
insmod part_bsd
insmod part_dfly
insmod part_dvh
insmod part_gpt
insmod part_msdos
insmod part_plan
insmod part_sun
insmod part_sunpc

loadfont /boot/grub/font.pf2
set gfxmode=800x600
set gfxpayload=keep

insmod efi_gop
insmod efi_uga

insmod all_video
insmod video_bochs
insmod video_cirrus
insmod gfxterm
insmod png
insmod jpeg
terminal_output gfxterm

insmod ext2
insmod f2fs
insmod ntfs
insmod exfat

insmod loopback
insmod iso9660
insmod udf

background_image /boot/splash.png
set timeout=10

loadfont /boot/grub/font.pf2


color_normal=cyan/black
#menu_color_highlight=black/light-gray
menu_color_highlight=yellow/red
menu_color_normal=light-gray/black

# First five entries usefully work with Ventoy

menuentry "${distname}-${revision} (${bootlabel} RAM0 no persistence)" {
 linux /vmlinuz w_bootfrom=/${distname}-${revision}.iso w_changes=RAM0 net.ifnames=0
 initrd /initrd.gz
}

menuentry "${distname}-${revision} (${bootlabel} RAM2 save on demand to ${savepart}_${savedir})" {
 linux /vmlinuz w_bootfrom=/${distname}-${revision}.iso w_changes=LABEL=${savepart}=/${savedir}/${distname}-${revision} w_changes1=RAM2 net.ifnames=0
 initrd /initrd.gz
}

menuentry "${distname}-${revision} (${bootlabel} direct saves to ${savepart}_${savedir})" {
 linux /vmlinuz w_bootfrom=/${distname}-${revision}.iso w_changes=LABEL=${savepart}=/${savedir}/${distname}-${revision} net.ifnames=0
 initrd /initrd.gz
}
  
menuentry "${distname}-${revision} (${bootlabel} RAM2 save on demand to ${bootlabel}_${savedir})" {
 linux /vmlinuz w_bootfrom=/${distname}-${revision}.iso w_changes=LABEL=${bootlabel}=/${savedir}/${distname}-${revision} w_changes1=RAM2 net.ifnames=0
 initrd /initrd.gz
}

menuentry "${distname}-${revision} (${bootlabel} direct saves to ${bootlabel}_${savedir})" {
 linux /vmlinuz w_bootfrom=/${distname}-${revision}.iso w_changes=LABEL=${bootlabel}=/${savedir}/${distname}-${revision} net.ifnames=0
 initrd /initrd.gz
}

menuentry "Shutdown" {
	halt
}

menuentry "Reboot" {
	reboot
}

this is the /boot directory in the ISO

Screenshot(55).jpg
Screenshot(55).jpg (18.08 KiB) Viewed 1059 times
Screenshot(56).jpg
Screenshot(56).jpg (18.28 KiB) Viewed 1059 times
gyrog
Posts: 594
Joined: Thu Oct 01, 2020 8:17 am
Location: Australia
Has thanked: 14 times
Been thanked: 180 times
Contact:

Re: Booting Puppy ISO files, enhanced loopback.cfg, utility to do it

Post by gyrog »

@rockedge, I'm not sure how to take your latest post.

So, I'm just going to try to clarify what I'm about in this forum topic.

1) While current woof-ce produces a working 'loopback.cfg', it could do more to take full advantage of the 'isoboot' script that is in 'initrd.gz'.
The 'mk-grub2-cfgs' utility is supposed to be a demonstration of how this might be implemented in 'woof-CE/woof-code/support/mk_iso.sh'.

2) There needs to be a utility that makes it easy to directly boot ISO files, utilising the grub2 loopback facility, via 'loopback.cfg'.
One that does only that. SG2D takes too many menu selections to get to it.
The 'mk-img-boot-entries' utility is a demonstration of one simple approach to such a utility, there must be others.
It also demonstrates how to generate a grub boot entry that takes full advantage of an "enhanced" Puppy 'loopback.cfg'.

Clarity
Posts: 3270
Joined: Fri Jul 24, 2020 10:59 pm
Has thanked: 1347 times
Been thanked: 438 times

Re: Booting Puppy ISO files, enhanced loopback.cfg, utility to do it

Post by Clarity »

I have been shadowing this thread. Its bottom line is to demonstrate approaches to ease the use of WoofCE productions, specifially, in running those distros directly from their ISO files.

It is obvious, to me, the advantages that is the object in directly running the ISO flie and maintenance of persistence in EXACTLY the same way that we understand what is known as Frugal. Further I believe, like me, the OP author feels that an outcome can occur which further reduces the problem seen over the past 2 decades where SFSs at boot are misaligned or found missing on the many various PCs the world's members are using or coming to Puppyland for migrations or testing.

The approach being displayed is remarkably similar to how @wiak approached Kernel Linux (KL) distros to become a download and run for the KL distro family. His work as well as the efforts here are addressing the varied ways to directly boot ISOs. And it appears that effort here will address what we currently can do in both KL land and DOG land distros, today.

@jamesbond and @kirk have seemingly saw the same advantage and built the abilities into FATDOG.

Developers-users of all kinds dont have to "install" to test forum distros as these efforts draw to conclusion. These "boot launch" efforts can be done from the multiple well-known vehicles covered across the forum. A brief summary for directly booting from distro ISO files are

  • QEMU and other VM services

  • Docker and similar tools like barryk's EASY

  • Directly via GRUB2 stanza

  • Download and launch via SG2D, ISObooter, Ventoy without modifications of the boot stanza(s)

The ISO-file boot experience is identical to installing and running forum distros...except the installation piece in not necessary for full benefit. Further, if someone wants, they can boot the ISO file and "install" directly from the booted distro's ISO file as that choice is available to anyone, too.

Finally, I hope to be forthcoming soon with a fool-proof manner for any new user to eliminate the various USB issues that we continue to see by eliminating the efforts users (and forum support personnel) run into with trying to write a USB stick with Lick, Etcher, Rufus, dd, etc. It is based upon an observation I have be looking over for the past year. I think this can be reduced to a simple 'singularity'.

"Fingers crossed that the community will find it an acceptable replacement to those pesky problems."

Edit: Spelling corrections done 23-11-22

Last edited by Clarity on Wed Nov 22, 2023 10:23 pm, edited 3 times in total.
User avatar
rockedge
Site Admin
Posts: 5711
Joined: Mon Dec 02, 2019 1:38 am
Location: Connecticut,U.S.A.
Has thanked: 1991 times
Been thanked: 2097 times
Contact:

Re: Booting Puppy ISO files, enhanced loopback.cfg, utility to do it

Post by rockedge »

@gyrog I'm kind of thinking out loud is all.

gyrog
Posts: 594
Joined: Thu Oct 01, 2020 8:17 am
Location: Australia
Has thanked: 14 times
Been thanked: 180 times
Contact:

Re: Booting Puppy ISO files, enhanced loopback.cfg, utility to do it

Post by gyrog »

gyrog wrote: Wed Nov 15, 2023 11:02 am

This utility should disappear if/when the appropriate changes are made to 'woof-code/support/mk_iso.sh'.
It has been created by extracting the relevant parts of 'mk_iso.sh' and changing them.
So it should be no problem to generate a woof-ce patch.

So I did a fork of woof-ce on github and patched '/woof-code/support/mk_iso.sh' with the changes implied by my 'mk-grub2-cfgs' utility.
I then ran woof to produce a new "patched" Puppy. The new ISO did include appropriate "patched" 'grub.cfg' and 'loopback.cfg' files.
The ISO file booted as expected using 'mk-img-bootentries', (no "waiting 5 seconds" message).
I then burnt the ISO to a DVD, and it booted fine.
So, the changes can be sucessfully applied to Woof-ce, and 'mk-grub2-cfgs' can go.

I've also tried bootiing it with "isobooter3", it booted directly to a desktop, with a"waiting 5 seconds" message.
This is because it does not use the 'loopback.cfg' present in the ISO.

While I had the ISO booted, I tried the 'BootFlash' utility to produce a bootable USB stick.
That also booted fine.

Clarity
Posts: 3270
Joined: Fri Jul 24, 2020 10:59 pm
Has thanked: 1347 times
Been thanked: 438 times

Re: Booting Puppy ISO files, enhanced loopback.cfg, utility to do it

Post by Clarity »

@gyrog
If you post your test ISO file somewhere, I will run it through the 5 known ISO file boot launch approaches and report back here.

gyrog
Posts: 594
Joined: Thu Oct 01, 2020 8:17 am
Location: Australia
Has thanked: 14 times
Been thanked: 180 times
Contact:

Re: Booting Puppy ISO files, enhanced loopback.cfg, utility to do it

Post by gyrog »

@Clarity, thanks for the offer.
I'll have to upload it which will take a while, it's huge, it's 830MiB.

I set out to woof a Jammypup32, since it is relatively small.
But I wasn't paying enough attention when I started the woof, and ended up with a Jammypup64.

I'll post a URL after I've uploaded it.

gyrog
Posts: 594
Joined: Thu Oct 01, 2020 8:17 am
Location: Australia
Has thanked: 14 times
Been thanked: 180 times
Contact:

Re: Booting Puppy ISO files, enhanced loopback.cfg, utility to do it

Post by gyrog »

For what it's worth, here is my test ISO file, generated by a patched woof-ce.
https://www.mediafire.com/file/77p617q9 ... 0.iso/file

Clarity
Posts: 3270
Joined: Fri Jul 24, 2020 10:59 pm
Has thanked: 1347 times
Been thanked: 438 times

Re: Booting Puppy ISO files, enhanced loopback.cfg, utility to do it

Post by Clarity »

I downloaded the ISO onto my Ventoy USB stick's BOOTISOS folder.

I have done 3 test cases. (I really should develop a test-case spreadsheet similar to @wizard's example). I have tested 3 individual cases booting the ISO file:

  • bare-metal via USB

  • QEMU via ISO file

  • QEMU via USB

Environment

  • INtel i5

  • 8GB RAM

  • nvidia HDMI via noveau

  • USB stick with approx 20+ ISO files in its BOOTISOS folder

Note for understanding:
For several years, now, I have use a Ventoy USB that has both SG2D as well as ISObooter on it. For WoofCE PUPs, and other pesky forum distros, I have Ventoy launch SG2D. There, I allow SG2D to list and launch WoofCE PUPs...as well as couple other forum ISO files. This single USB gives me the ability to test-use all ISO file boot-launchers; namely Ventoy, SG2D, ISObooter, and PXE. It has been convenient for test the booting of these forum distros. allowing me to help as many forum member as I can to ease many of the issues they run into.

Bare Metal scenario:

  1. Booted the USB and launched SG2D taking the normal path to SG2D's menu.

  2. Hit SG2D's default menu to get the ISO files listing.

  3. Scrolled to jammyPUP64gyrog and launched it ... taking its Menu default selection without modification

  4. System boots to desktop where FirstRun gets local changes

  5. Minor desktop changes are made and several apps navigated, including its browser

  6. Reboot done saving its session in the folder location that SAVESPEC identifies

  7. REBOOT taking the same path where, this time, the PUP finds its session automatically as it proceeds to desktop.

Mission Accomplished without isses

QEMU via ISO file

  1. Booted PC to BKWPUP64's desktop

  2. Opened QEMU VM for booting the ISO file directly. As mentioned before, I had downloaded the ISO into a BOOTISOS folder.

    Screenshot.jpg
    Screenshot.jpg (27.84 KiB) Viewed 831 times
  3. The VM, for this test, does NOT contain a persistence partition

    Screenshot(1).jpg
    Screenshot(1).jpg (32.62 KiB) Viewed 831 times
  4. ISO proceeds to desktop.

Mission Accomplished without issues

QEMU via Ventoy USB
Note: In this test, the BOOTISOS folder also contains the SAVESPEC even though there will be no persistence partition in the VM's setup stanza. This may be of importance as there is a point during the boot that this MAY be a factor.

  1. Opened QEMU VM for booting the ISO file out of the USB. As mentioned before, I had downloaded the ISO into a BOOTISOS folder.

  2. The VM, for this test, does NOT contain a persistence partition

    Code: Select all

    qemu-system-x86_64 -name "JammyPUP64 v10 by Gyrog" -enable-kvm -smp 2 -m 2048 -vga std -device AC97 -net nic -net user -rtc base=localtime -hda /dev/sdg
  3. booting seems to struggle as it goes on a hunt for persistence (I assume this is normal when there is a SAVESPEC present)

    Screenshot(5).jpg
    Screenshot(5).jpg (47.87 KiB) Viewed 828 times
  4. After a period of time, it gives up and the ISO proceeds to desktop.

Mission Accomplished without issues

Additional items of NOTE
There are 2 other test I have not done and can report on them later:

  • QEMU booting a SG2D USB instead of a Ventoy

  • bare-metal SG2D CD launching the ISO file residing in a bootisos folder on the system drive

These would cover all the known ways that users and developer might use to test-run by launching a forum's ISO file(s) for normal desktop operations.

Using the launchers, I have no reason to ever install a forum distro (or the many distros found across the Distrowatch world either, including debian/ubuntus and much more).
Reason: The forum distros ISOs not only give us desktops, but, they also give us persistence. Thus I have a complete system without the headaches that many constantly need assistance..directly from a single USB. The importance is that the USB is made only once and maintained by merely added/removing ISO files to its BOOTISOS folder, over time-forever.

Clarity
Posts: 3270
Joined: Fri Jul 24, 2020 10:59 pm
Has thanked: 1347 times
Been thanked: 438 times

Re: Booting Puppy ISO files, enhanced loopback.cfg, utility to do it

Post by Clarity »

One more test completed:
Booting a CD in a QEMU VM:

  1. Booted the CDROM drive with a SG2D CD within.

    Code: Select all

    qemu-system-x86_64 -boot d -m 2048 -cdrom /dev/cdrom -smp 2 -vga std -device AC97 -net nic -net user -rtc base=localtime -hda /dev/sdg
    Screenshot(3).jpg
    Screenshot(3).jpg (53.61 KiB) Viewed 798 times
  2. Selected the jammyPUP64 from the ist of ISOs it found and desktop emerges, as well.

Screenshot(6).jpg
Screenshot(6).jpg (28.62 KiB) Viewed 803 times

Mission Accomplished without isses

FYI

gyrog
Posts: 594
Joined: Thu Oct 01, 2020 8:17 am
Location: Australia
Has thanked: 14 times
Been thanked: 180 times
Contact:

Re: Booting Puppy ISO files, enhanced loopback.cfg, utility to do it

Post by gyrog »

@Clarity,
Thanks for testing.

No surprises.

SG2D uses the grub2's loopback facility and the 'loopback.cfg' in the ISO file.
Puppy sees this as an ISO boot and uses the SAVESPEC in the directory containing the ISO file.

Ventoy does NOT use grub2's loopback facility.
Puppy sees it as a boot from an actual CD, hence there is no SAVESPEC to process.
If you try to save, it will try to use the 'ventoy' device which is some kind of pseudo-CD.
"save"ing would actually work better if Ventoy did not insert a "pmedia=" line into the 'init' script, overriding the appropriate "pmedia=cd".

Edit:
On reading your results again, it looks like you never tried to use Ventoy to directly boot the test ISO file, so my comments re Ventoy are not really relevant.

Your "QEMU via ISO file" test is what is to be expected when booting an actual CD burnt from the ISO.
The 'init' script does not see this as needing the 'isoboot' script, so no SAVESPEC is used.

Clarity
Posts: 3270
Joined: Fri Jul 24, 2020 10:59 pm
Has thanked: 1347 times
Been thanked: 438 times

Re: Booting Puppy ISO files, enhanced loopback.cfg, utility to do it

Post by Clarity »

gyrog wrote: Sat Nov 25, 2023 4:15 pm

... you never tried to use Ventoy to directly boot the test ISO file...

I did not test Ventoy booting the ISO file for the very facts you share about how WoofCE PUPs see their environment!. And, as well as the resulting issues presented on the BKW64 thread.

I should test, though.

... Your "QEMU via ISO file" test is what is to be expected when booting an actual CD burnt from the ISO.
The 'init' script does not see this as needing the 'isoboot' script, so no SAVESPEC is used.

I wonder if you might reconsider this point for future.

SG2D, as you know, hunts the storage medias for folder(s) with eligible ISO files within, as he builds the "launch list" for user selection. A SAVESPEC would still be applicable to directing to persistence wanted by users. This would negate any need for a user to manually adjust ISO file's boot menu stanza parms, differently, than what happens when launched from the USBs.

Just an idea. If reasonable you know...

Lastly, I think you also know that the SG2D author has a GIT site that he routinely visits. He support it different from Ventoy which has a forum page for support.

Clarity
Posts: 3270
Joined: Fri Jul 24, 2020 10:59 pm
Has thanked: 1347 times
Been thanked: 438 times

Re: Booting Puppy ISO files, enhanced loopback.cfg, utility to do it

Post by Clarity »

BTW: Did you know that your PUP test-distro performs very well?

gyrog
Posts: 594
Joined: Thu Oct 01, 2020 8:17 am
Location: Australia
Has thanked: 14 times
Been thanked: 180 times
Contact:

Re: Booting Puppy ISO files, enhanced loopback.cfg, utility to do it

Post by gyrog »

Clarity wrote: Sat Nov 25, 2023 10:42 pm

I wonder if you might reconsider this point for future.

SG2D, as you know, hunts the storage medias for folder(s) with eligible ISO files within, as he builds the "launch list" for user selection. A SAVESPEC would still be applicable to directing to persistence wanted by users. This would negate any need for a user to manually adjust ISO file's boot menu stanza parms, differently, than what happens when launched from the USBs.

Just an idea. If reasonable you know...

When you boot an ISO as a file, there has to be at least a boot parameter that idetifies the relative path of the ISO file, and in the best case, also a boot parameter that identifies the partition that contains the ISO file.
The presence of any of these boot parameters is detected by the 'isoboot' script, which has to establish the location of the ISO file.
The directory containing the ISO file is a writable location, which is appropriate for a SAVESPEC.

When booting from a CD, or an ISO dd'ed to a usb stick, or a QEMU setup as a CD/DVD, there are no boot parameters to identify the location of the source iso9660. So Puppy has to "search" for it. Even when it is located all we have is a RO filesystem, there is no connection to any particular writable location on the computer. So there is nowhere for you to write a SAVESPEC that 'iinit' can connect to the Puppy you are booting. The only option for 'init' is to "search" "all" the partitions looking for a likely savefolder/savefile, which it does when "pmedia=cd".
And there is no point in having 'init' searching for a SAVESPEC, instead of seraching for savefolder/savefile.

For a SAVESPEC to be useful there would need to be a quick way for the 'init' script to locate it.
Currently, when booting a CD/DVD, (or equivalent), there is no way.

gyrog
Posts: 594
Joined: Thu Oct 01, 2020 8:17 am
Location: Australia
Has thanked: 14 times
Been thanked: 180 times
Contact:

Re: Booting Puppy ISO files, enhanced loopback.cfg, utility to do it

Post by gyrog »

Clarity wrote: Sat Nov 25, 2023 10:42 pm

Lastly, I think you also know that the SG2D author has a GIT site that he routinely visits. He support it different from Ventoy which has a forum page for support.

No. Please provide a URL.

Post Reply

Return to “Cutting Edge”