Packaging using a local repository

saitam

Hi. I 'm trying to package a larger collection of dependant libraries. What I read so far, the local repository solution is the way to go, but I ran into some problems. At first, I uninstalled solbuild-config-unstable and installed solbuild-config-local-unstable. Also, I want to build against the stable repository. I followed the steps described in this discussion. The interesting part is at the end. Running sudo solbuild build -p main-x86_64 (also in the version main-x86_64.local) I get the message
`Error: 'main-x86_64.local' is not a known profile
Valid profiles include:

Error loading profiles: Cannot enable unknown repo Local
`
Does anybody has a hint what I'm doing wrong ?

Girtablulu

You still need the unstable config

saitam

Ok, I installed solbuild-config-unstable again, run sudo solbuild init , and sudo solbuild build -p main-x86_64 again. Still the same error. By the way, in 00_solbuild.conf it is set default_profile = "main-x86_64".

Girtablulu

just run make local

saitam

Girtablulu Now I'm one step further and encountered the next problem:

The first library was successfully compiled. The resulting .eopk and -devel.eopkg were copied to /var/lib/solbuild/local according to this description

Then I changed into the directory /var/lib/solbuild/local and run sudo solbuild index -p main-x86_64 to add the new package to the local stable repository index. Seems to be working, resulting output is

Adding package to index: numactl-2.0.12-1-1-x86_64.eopkg                        
Index file written

Started compiling the second library. Without any defined builddeps in the second package.yml file, a header file contained in the first package is not found. So I wanted to add the first library as a dependency in the second package.yml. Here I run into problems. According to the first package name numactl-2.0.12-1-1-x86_64.eopkg I tried
pkgconfig(numactl), pkgconfig(numactl-2.0.12), pkgconfig(libnuma) and here I get

[pkgconfig:libnuma] This dependency is not in any repo [BuildDep] pkgconfig(libnuma) build dep does not exist in the repository.

Justin

saitam Step 2 isn't needed. When using make local you don't need to index it, it does it for you.

For numactl run eopkg info numactldevelfile.eopkg and it should tell you what the pkgconfig name is.

saitam

Justin For some reason my solbuild configuration was not correct. I reinstalled it and with your hint using eopkg info I got the second library compiled. Thank you. Because this library provides no easy installation mechanism (like make install ), I have to copy manually the resulting lib.a, lib.so and header files to their destination folders. But for doing this I have to further read the packaging documentation. If I get stuck, I will call back 😃

kyrios

Usually we nuke the static libraries (.a files) and we only keep the shared libraries (.so files)

saitam

kyrios In the install section of the package.yml file I defined the destinations of library and header files and I get an .eopkg file after running make local. lseopkg shows the correct structure afterwards. But I get no -devel.eopkg file. Is this correct or did I something wrong ?

kyrios

saitam -devel is created only when there are development libraries or headers (or if you force it with patterns). So yeah it can be perfectly normal to have no -devel package created.

saitam

kyrios You wrote -devel is created only when there are development libraries or header. This is the case for library I want to create a package from. A further hint that I did something wrong is that eopkg info yields no Provides: pkgconfig() entry that I can use in dependent builds. Here is the install section of the package.yml. Maybe you can identify the error.

install    : |
    cd build
    install -D -m 00755 libhsakmt.so.1.0.0 $installdir/opt/rocm/lib/libhsakmt.so.1.0.0
    ln -s $installdir/opt/rocm/lib/libhsakmt.so.1.0.0 $installdir/opt/rocm/lib/libhsakmt.so.1
    ln -s $installdir/opt/rocm/lib/libhsakmt.so.1.0.0 $installdir/opt/rocm/lib/libhsakmt.so

    echo "/opt/rocm/lib" > x86_64-libhsakmt.conf
    install -Dm00644 x86_64-libhsakmt.conf $installdir/usr/share/ld.so.conf.d/x86_64-libhsakmt.conf

    cp -r hsakmt-roct-dev $installdir/opt/rocm

    cd ..
    cp -r include $installdir/opt/rocm

kyrios

Not all packages provides pkgconfig (.pc file(s)). Also Solus doesn't ship /opt by default instead we install libs in %libdir% or %libdir%/rocm and other ressources in /usr/share/rocm for example.

saitam

kyrios Thank you for your answer. This weird installation path is common in the ROCm library stack and people are complaining about this. But I'll try how far I will come with the locations you suggest. I changed now the package.yml file according to your suggestions:

install -D -m 00644 libhsakmt.so.1.0.0 $installdir/%libdir%/rocm/libhsakmt.so.1.0.0
    ln -s $installdir/%libdir%/libhsakmt.so.1.0.0 $installdir/%libdir%/rocm/libhsakmt.so.1
    ln -s $installdir/%libdir%/libhsakmt.so.1.0.0 $installdir/%libdir%/rocm/libhsakmt.so

    cd ..
    install -Dm00644 include/hsakmt.h $installdir/usr/include/rocm/hsakmt.h
    install -Dm00644 include/hsakmttypes.h $installdir/usr/include/rocm/hsakmttypes.h
    install -Dm00644 include/linux/kfd_ioctl.h $installdir/usr/include/rocm/linux/kfd_ioctl.h

Now I also get a *.eopkg file containing the libraries and a devel.eopkg file containing the headers. I will try if I can proceed with this.

saitam

kyrios I'm still trying to do the packing of ROCm stuff and still run into problems. For better basis for discussion I uploaded the package.yml's to github. In the Readme you also find the results of lseopkg. There is the following problem:

Package ROCT-Thunk-Interface install among other things the library /usr/lib64/rocm/lib64/libhsakmt.so.1
Package ROCR-Runtime builds and creates an eopkg package but throws an error Fatal: Unknown symbol: libhsakmt.so.1
This missing libhsakmt.so.1 is afterwards causing errors when building libraries dependant on ROCR-Runtime
lseopkg shows correct installed libraries for ROCT-Thunk-Interface and if I install the ROCT-Thunk-Interface eopkg package locally and compile ROCR-Runtime manually everything works.

If you take a look on the package.yml files in my git repo, do you find any error I made ? Thank you very much.

saitam

The last days I made some improvements in my experimental packaging project and I managed to obtain some eopkg packages. But now I ran into problems. I trying to package the ROCm compute stack for AMD GPUs. I read that DataDrake is working on this and when he has failed so far, I will definitely fail, too 😀. But at least I learn something. The problems I currently have are more related to the weird project structure of the ROCm stuff. As @kyrios mentioned before I put libraries into /usr/lib64/rocm and include files into /usr/include/rocm. Unfortunately, later dependant programs needs a project structure for the build process that cannot be changed with cmake parameters. And the ROCm default project path is /opt/rocm. My plan was to put the libraries and headers into the correct /usr/lib and /usr/include folders and just create symbolic links to /opt/rocm to finish the building and deleting the /opt/rocm folders afterwards.

First question: Is this a feasible approach ? Otherwise I can stop here my packaging experiment.
I have problems creating symbolic links of folders. Libraries are located in the /usr/lib64/rocm/ folder in the package.yml I have
mkdir -p $installdir/opt/rocm ln -s /usr/lib64/rocm $installdir/opt/rocm/lib
I get the error message [Errno 2] No such file or directory: '/home/build/YPKG/root/ROCT/install/opt/rocm/lib'

Thank you.

kyrios

saitam sorry I am on holiday with just my smartphone and no computer so I only answer the basic things. And if DataDrake is working on it, it means it's beyond my skills. 😁

saitam

kyrios Thank you for your answer. I will post the question as a new discussion, more related to this special topic. Enjoy your holiday !

brent

kyrios I have two holiday strategies, depending on mood. 1) No phone at all. All day. and 2) "All Sounds Off."
Enjoy your time, K.
(Forgive me for off-topic, just wanted to send best wishes.)