is there an apt-file-like functionality in eopkg?

ender · Feb 9, 2020

Relevant discussion I guess: https://discuss.getsol.us/d/3617-how-to-find-a-package-that-provides-x which would amount to “not yet, maybe later”(to my understanding)

hyphens · Feb 11, 2020

I've spent some time in the package-management source tree, and unpacking packagedb, filesdb, etc..
From what I gathered, it looks like the list of files for each package is not embedded in the repo's index.xml. Rather, it is read from an installed package's files.xml with each install operation.

filesdb, the file eopkg sf uses, is a shelved python dict that takes this structure: { 'path md5 digest': 'package name' }. Structuring a larger db like this but for all packages in the repo should be trivial, but the real issue would be the need to fetch and unpack the entirety of Shannon's 10k+ packages and reading the files.xml for each package :c

I had the intentions to write either a patch or an external tool to introduce this functionality but looks like I'm stopped in my tracks -- now that I understand the repo infrastructure.

hyphens · Feb 11, 2020

the only way I can imagine around this is that the db itself gets constructed on the repo side and packaged (it will probably be around 200MB in size, assuming that size:npackages is directly proportionate and my files.db is 17MB for ~ 900 packages). it would get updated with each sync and would probably install a file that directly replaces files.db
or like datadrake said, a webservice

DataDrake · Feb 11, 2020

hyphens I wouldn't be using Python for it. I once wrote a replacement for mlocate that parsed something like 2.5 million local files and could store all of that in < 10MB uncompressed and something like < 5MB compressed. I'm pretty sure I can do better nowadays. I just don't want to worry about it until sol is finished. And yeah, it would probably get generated server-side by ferryd like the package index, except only enabled optionally.

Ttele · Mar 23, 2021

$ inxi -S
System:    Host: home Kernel: 5.11.6-174.current x86_64 bits: 64 Desktop: MATE 1.24.1 Distro: Solus 4.2

dpkgS

#!/bin/bash

#echo $1


#	repo:		https://mirrors.rit.edu/solus/packages/shannon/eopkg-index.xml.xz
#	path to ipatbles:   <PackageURI>i/iptables/iptables-1.6.2-7-1-x86_64.eopkg</PackageURI>
#	main path:		https://mirrors.rit.edu/solus/packages/shannon/

MAIN_PATH_OF_REPO="https://mirrors.rit.edu/solus/packages/shannon/"
PATHS_OF_PACKAGES=$(grep PackageURI /var/lib/eopkg/index/Solus/eopkg-index.xml | grep "$1" | sed -e 's/<[^>]*>//g')

while read PATH_OF_PACKAGE; do
	PACKAGE_NAME=$(echo $PATH_OF_PACKAGE | sed -e 's/.*\///g' | rev | cut -d"-" -f 5- | rev)
	#echo $PACKAGE_NAME
	if [ "$1" == "$PACKAGE_NAME" ] ; then
		echo "PACKAGE_NAME = $PACKAGE_NAME"
		FILE_NAME=$(echo $PATH_OF_PACKAGE | sed -e 's/.*\///g')
		DOWNLOAD=$(curl -s --show-error -O "${MAIN_PATH_OF_REPO}${PATH_OF_PACKAGE}")
		RESULT=$?
		[ ! $RESULT -eq 0 ] && echo "$DOWNLOAD"
		unzip -p "$FILE_NAME" files.xml > files.xml
		grep Path files.xml | sed -e 's/<[^>]*>//g'  | awk '{ print "/" $1 }'
		rm "$FILE_NAME"
		rm files.xml
	fi
done <<< "$PATHS_OF_PACKAGES"

Ttele · Mar 23, 2021

$ eopkg sf iptables
Searching for iptables
Package bash-completion has file /usr/share/bash-completion/completions/iptables
Package iptables has file /sbin/iptables
Package iptables has file /sbin/iptables-restore
Package iptables has file /sbin/iptables-save
Package iptables has file /usr/bin/iptables-xml
Package iptables has file /usr/share/man/man1/iptables-xml.1
Package iptables has file /usr/share/man/man8/iptables-extensions.8
Package iptables has file /usr/share/man/man8/iptables-restore.8
Package iptables has file /usr/share/man/man8/iptables-save.8
Package iptables has file /usr/share/man/man8/iptables.8

$ bash dpkgS iptables
PACKAGE_NAME = iptables
/lib64/xtables/libebt_802_3.so
/lib64/xtables/libebt_ip.so
/lib64/xtables/libebt_log.so
/lib64/xtables/libebt_mark_m.so
/lib64/xtables/libip6t_DNAT.so
/lib64/xtables/libip6t_DNPT.so
/lib64/xtables/libip6t_HL.so
/lib64/xtables/libip6t_LOG.so
/lib64/xtables/libip6t_MASQUERADE.so
...

Output of dpkgS is much longer.

DataDrake · Mar 23, 2021

tele That's not the right command. It's eopkg info -F <package-name>.

Ttele · Mar 23, 2021

DataDrake
I don't know what it is sol, but I hope something like this will happen someday.

You can create script to create database on server.
You can add script to crontab or to app / script when repository trying update eopkg-index.xml on server.
Then we can try download and use this database to list files of package
or to find which package provide some kind file.
You can do for example
/database/i/iptables/iptables-1.6.2-7-1-x86_64.eopkg
where instead eopkg will list of files which provide this package.
Then we can try use grep -ri word to search.

Ttele · Mar 24, 2021

DataDrake
Ok. Thanks.

DataDrake · Mar 24, 2021

tele sol is the eventual replacement for eopkg.

I wouldn't use a database.
This is managed by ferryd which will eventually provide the server-side functionality you are expecting.
There was always going to be something like this, but it will be a single heavily-compressed file that will be opt-in and fetched when updating the package index.

Ttele · Mar 24, 2021

Thank you very much for your explanation.

You're right. Whether a folder or a file will, it must be compressed before we download.
on the same principle works files.xml which is compressed to eopkg-index.xml.xz
Please only don't pack in the same file.
This will result in slower upgrades on less powerful hardware.
I suggest a separate file and download only with a specific command.
You can also add a condition.
```
If the file from today exists, use that file.
Otherwise, download a new file.
```
You can add the option "force download new file" .

Off topic:

I have not noticed eopkg info -F <package-name> before because in eopkg --help
I don't see that eopkg has any more options.
If there was a suggestion before then I would see man eopkg

Solus have problem with files

tele@home ~/Documents $ ls 
dpkgS
tele@home ~/Documents $ bash dpkgS iptables > iptables-1.6.2-7-1-x86_64.eopkg
tele@home ~/Documents $ ls
dpkgS
tele@home ~/Documents $ bash dpkgS iptables > file.txt
tele@home ~/Documents $ ls 
dpkgS  file.txt
tele@home ~/Documents $ touch iptables-1.6.2-7-1-x86_64.eopkg
tele@home ~/Documents $ ls 
dpkgS  file.txt  iptables-1.6.2-7-1-x86_64.eopkg

If the file is created by some miracle (I shortened the file name a little)
and I open this file in the package manager, I have a message that I cannot open the file (correct)
But the file can disappear after this.
Maybe it's just my case, I don't know.

That's all, have a nice day

DataDrake · Mar 24, 2021

tele Please only don't pack in the same file.
This will result in slower upgrades on less powerful hardware.

You say this like I haven't considered that. I'm not a moron. That's the reason it's opt-in and not part of the index itself.

tele I suggest a separate file and download only with a specific command.

That defeats the purpose of it. You would have to know which file to download in order to even look for something. At which point you probably would have it installed already and would already have the files list.

tele I have not noticed eopkg info -F <package-name> before because in eopkg --help
I don't see that eopkg has any more options.
If there was a suggestion before then I would see man eopkg

It's listed if you look at eopkg help info. It's also in the manpage under the info sub-command.

tele If the file is created by some miracle (I shortened the file name a little)
and I open this file in the package manager, I have a message that I cannot open the file (correct)
But the file can disappear after this.
Maybe it's just my case, I don't know.

I have no idea what you are trying to do with those commands. You shouldn't be trying to make an .eopkg like that.