Short version:
Kernels published by Linux Kernel Foundation include the following top-level folders: lib, lib32 and lib64 and the infra-structure enabling their employment by applications. The recent ‘usr-merge’ Rule –adopted by some Major distros-- does not preclude the use of those folders, but rather requires it. Puppys’ modifications to Kernels do not change the above. Puppys’ initrds do not modify that.
I had had success with peebee’s 64bit Compatibility SFS under several 32-bit Puppys. But it was reported that the companion 32bit SFS for use under 64-bit Puppys did not work under Voidpup64 ‘do NOT use on VoidPup64 due to its lib structure’ https://www.forum.puppylinux.com/viewto ... 441#p78294. IIRC, I then tried the 64bit SFS under VanillaDpup and while no notice of a problem was displayed, no applications I tried to run did so. [Maybe I just didn’t wait long enough]. In any event, I decided to explore what structures both 32-and-64bit Puppys were using (and how, i.e, symlinks as mentioned below). After deconstructing a half-dozen Puppys, I could confirm what I had suspected: There were 13 folders into which libraries could be placed, which of those folders existed in a particular Puppy depended on the architecture and flavor (I.e, 32/64bit; slackware, debian, ubuntu, void) of that Puppy and which folder was used depended on that Puppy. The word ‘Chaos’ comes to mind.
When there were only 32bit Puppys, there were only 4 locations into which libraries could be located: /lib, /usr/lib and /usr/local/lib* with Puppy’s unique 4th /root/my-applications/lib. When both Slackware and Ubuntu published 64bit OSes while continuing to publish 32bit OSes they forked on where to locate libraries. Slackware placed them in folders –top-level and under usr/ and /usr/local-- named lib64, while Ubuntu** placed them in similar folders named /x86_64-linux-gnu. As Puppys could be woofed from either Slackware or Ubuntu binaries, both structures had to be accommodated. Pets and SFSes could be created from Slackware packages OOTB, but libraries which Ubuntu placed in a folder named /x86_64-linux-gnu had to be moved to a /lib folder with a symbolic link named /x86_64-linux-gnu to that folder.
The implementation of the ‘usr-merge’ Rule created additional complications. I thought the Rule meant that user-created applications could NOT use top-level folders: only folders under /usr could be used.
After wasting a day trying the ‘move-libs’ technique –remember I had misinterpreted the ‘usr-merge’ Rule and that misinterpreted Rule doesn’t apply to Puppy’s unique /root/my-applications/lib folder-- I ‘googled’ to find out if there was a priority associated with the use of lib folders.
Which uncovered, Linux Filesystem Hierarchy Standard (FHS), https://www.linuxfordevices.com/tutoria ... -hierarchy.
That post revealed both the history behind locating files in /usr and the current structure of kernels build by The Linux Kernel Organization. In pertinent part it explains that the original reason for locating binaries –-the executables and their dependent libraries-- in /usr was a size limitation:
“The need for moving non-essential binaries to a different folder historically arose from a lack of space in the /bin hard disk. However, that was 1971. Today over 50 years later, we no longer face the same size problems. This has rendered two separate folders for default and user-installed binaries useless. Over time this has also caused a hodge-podge in the filesystems, with both the directory having redundant binaries which makes it confusing.”
It also explains how, in constructing a kernel, that Organization handles 32 and 64 bit libraries:
“A lot of times, there are different 32-bit and 64-bit libraries with the same name. To avoid any collusion, --sic, I think what was meant was ‘confusion’--these binaries are kept in two separate directories accordingly named /lib32 and /lib64.”
That post may be misinformed or I may have misunderstood it. But it explains why pets & SFSes for 64bit Slackos could be created without having to move libraries: the kernel already supported the use of /lib64 folders. It also would explain why my rebuilding of peebee’s SFS has worked all the 32bit Puppys I’ve tested:
All I did was:
1) Mount peebee’s SFS and copy its contents into a folder named 64bit-Compat.
2) Rename the Top-level lib folder lib64.
3) MOVE the contents of the /usr/lib folder into that /lib64 folder, over-writing its contents thereby eliminating duplication (the result is 3 Mbs smaller) and locating all non-duplicates in /lib64.
3a) Which left ‘symbolic-links’ which couldn’t be moved or created some broken symlinks. It is likely that almost all symlinks could be deleted but I wasn’t certain. Most symlinks were just to the libraries now in lib64***. But although that proved not be be the case, a symlink might have born the name of a file not in /lib64 but to a newer version of that library directing applications dependent on the older version to the newer one. Two symlinks in /usr/lib pointed to libraries having entirely different names. Sorry, I don’t remember which two. Regardless of what was likely, I recreated all necessary symlinks that were in /usr/lib in /lib64. This was the only PITA.
4) dir2sfs the 64bit-Compat folder (then remove the version number designation of the Puppy used to dir2sfs).
If the above analysis is correct, it should be possible to create a universal 32-bit Compatibility SFS by locating its libraries in a Top-level folder named lib32. My next project.
Edit: 
Well, as I wrote, as far as I can tell the 64bit Compatibility SFS works under all Puppys (at least if a recent kernel is used). But there must be something wrong with my theory or my implementation of it. Building a 32bit Compatibility SFS with all libraries in top-level /lib32 doesn't create a viable system. It loads. But applications don't run under it. Using either peebee's kernel for S15pup64 or rockedges kernel for F-96, attempts to run 32bit applications via a terminal displays a notice of 'wrong Elf class'. Yes, I tried restarting 'x' and even rebooting creating a SaveFolder.
While the notice that peebee's 32bit compatibility SFS doesn't work under Voidpup64 still stands, I'll take another look at both of his compatibility SFSes and see if that provides a clue. But, don't hold your breath. I have other fish to fry. Feel free to explore on your own.
-=-=-=--
* According to Linux Standards, /usr/local/ is "... used for all packages which are compiled manually from the source by the system administrator." Puppys don't honor that Rule:
- Bionicpup's _usr_local.png (64.04 KiB) Viewed 1340 times
** I don't know why Ubuntu chose to name folders /x86_64-linux-gnu. Maybe its devs didn't realize the kernel could already handle the dichotomy?
*** My recollection was that broken symlinks were left in /usr/lib. But, now working on a 32bit Compat SFS, moving files from /usr/lib to /lib32 created broken symlinks in /lib32 as the symlinks from /usr/lib over-wrote the actual libs in /lib32. I mounted the original SFS again to obtain another copy of the actual files. Nice that broken symlinks show up in red.
Two broken symlinks remained. But they already existed in the source SFS. Both point to folders to be provided by the operating system, the pcmcia folder in /etc and the other, ultimately, to the terminfo folder in lib.