All-Sky Gravitational Wave search on O3 v1.07 tasks compilation fail with ld.lld: error: undefined symbol: __printf_alloc

reanimator
reanimator
Joined: 14 Feb 23
Posts: 10
Credit: 68570460
RAC: 562124
Topic 231746

Running FC41 with clang18-libs package installed, compilation of ASGW tasks fails due to missing symbol:

ld.lld: error: undefined symbol: __printf_alloc

 

Here is an example of failed compilation job:

All-Sky Gravitational Wave search on O3 v1.07 (GW-opencl-ati-2)

https://einsteinathome.org/task/1688515297

 

The binary search works fine however:

Binary Radio Pulsar Search (MeerKAT) v0.17 (BRP7-opencl-ati)

https://einsteinathome.org/task/1683420145

 

Any suggestion what could be wrong here?

How to debug such issues?

Ian&Steve C.
Ian&Steve C.
Joined: 19 Jan 20
Posts: 4044
Credit: 48018441685
RAC: 35437838

What GPU? What drivers do you

What drivers do you have installed? Your host(s) is hidden on the website so we have no way to inspect the host details or more details from the failed tasks that might be helpful. 

_________________________________________________________________________

reanimator
reanimator
Joined: 14 Feb 23
Posts: 10
Credit: 68570460
RAC: 562124

Task

Task 1688515297

...

Failed OpenCL buildlog:
 

ld.lld: error: undefined symbol: __printf_alloc


>>> referenced by /tmp/comgr-6ce7dc/input/linked.bc.o:(XLALLoopOverCoarseGridFrequencyBins)
>>> referenced by /tmp/comgr-6ce7dc/input/linked.bc.o:(XLALLoopOverCoarseGridFrequencyBins)
Error: Creating the executable from LLVM IRs failed.


XLAL Error - XLALOpenCLGetProgramFromSource (/home/jenkins/workspace/workspace/EaH-GW-OpenCL-Testing/SLAVE/LIBC215/TARGET/linux-x86_64/EinsteinAtHome/source/lalsuite/lalpulsar/lib/GPUUtils/OpenCLUtils.c:705): clBuildProgram failed with OpenCL error: CL_BUILD_PROGRAM_FAILURE
XLAL Error - XLALOpenCLGetProgramFromSource (/home/jenkins/workspace/workspace/EaH-GW-OpenCL-Testing/SLAVE/LIBC215/TARGET/linux-x86_64/EinsteinAtHome/source/lalsuite/lalpulsar/lib/GPUUtils/OpenCLUtils.c:705): Generic failure
XLAL Error - XLALGCTOpenCLKernelsSetup (/home/jenkins/workspace/workspace/EaH-GW-OpenCL-Testing/SLAVE/LIBC215/TARGET/linux-x86_64/EinsteinAtHome/source/lalsuite/lalapps/src/pulsar/GCT/HierarchSearchGCT_OpenCL.c:212): Check failed: XLALOpenCLGetProgramFromSource ( source, &(GCTOpenCLKernels.HierarchSearchGCTProgramm) ) == XLAL_SUCCESS
XLAL Error - XLALGCTOpenCLKernelsSetup (/home/jenkins/workspace/workspace/EaH-GW-OpenCL-Testing/SLAVE/LIBC215/TARGET/linux-x86_64/EinsteinAtHome/source/lalsuite/lalapps/src/pulsar/GCT/HierarchSearchGCT_OpenCL.c:212): Internal function call failed: Generic failure
XLAL Error - MAIN (/home/jenkins/workspace/workspace/EaH-GW-OpenCL-Testing/SLAVE/LIBC215/TARGET/linux-x86_64/EinsteinAtHome/source/lalsuite/lalapps/src/pulsar/GCT/HierarchSearchGCT.c:1394): Check failed: XLALGCTOpenCLKernelsSetup( uvar->SortToplist, uvar->getMaxFperSeg, uvar->computeBSGL, detectorIDs, usefulParams.BSGLsetup ) == XLAL_SUCCESS
XLAL Error - MAIN (/home/jenkins/workspace/workspace/EaH-GW-OpenCL-Testing/SLAVE/LIBC215/TARGET/linux-x86_64/EinsteinAtHome/source/lalsuite/lalapps/src/pulsar/GCT/HierarchSearchGCT.c:1394): Internal function call failed: Generic failure
2024-11-28 13:00:47.2900 (131814) [CRITICAL]: ERROR: MAIN() returned with error '-1'
Code-version: %% LAL: 7.1.4.1 (CLEAN )
%% LALPulsar: 3.1.0.1 (CLEAN )
%% LALApps: 7.3.0.1 (CLEAN )

...

reanimator
reanimator
Joined: 14 Feb 23
Posts: 10
Credit: 68570460
RAC: 562124

What drivers do you have

What drivers do you have installed?

 

modinfo amdgpu

name: amdgpu
vermagic: 6.11.8-300.fc41.x86_64 SMP preempt mod_unload


rocm-clinfo

 

Number of platforms:                 2
  Platform Profile:                 FULL_PROFILE
  Platform Version:                 OpenCL 3.0 CUDA 12.7.33
  Platform Name:                 NVIDIA CUDA
  Platform Vendor:                 NVIDIA Corporation
  Platform Extensions:                 cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_fp64 cl_khr_3d_image_writes cl_khr_byte_addressable_store cl_khr_icd cl_khr_gl_sharing cl_nv_compiler_options cl_nv_device_attribute_query cl_nv_pragma_unroll cl_nv_copy_opts cl_khr_gl_event cl_nv_create_buffer cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_nv_kernel_attribute cl_khr_device_uuid cl_khr_pci_bus_info cl_khr_external_semaphore cl_khr_external_memory cl_khr_external_semaphore_opaque_fd cl_khr_external_memory_opaque_fd cl_khr_semaphore
  Platform Profile:                 FULL_PROFILE
  Platform Version:                 OpenCL 2.1 AMD-APP (3625.0)
  Platform Name:                 AMD Accelerated Parallel Processing
  Platform Vendor:                 Advanced Micro Devices, Inc.
  Platform Extensions:                 cl_khr_icd cl_amd_event_callback
  Platform Name:                 AMD Accelerated Parallel Processing
Number of devices:                 1
  Device Type:                     CL_DEVICE_TYPE_GPU
  Vendor ID:                     1002h
  Board name:                     AMD Radeon VII
  Device Topology:                 PCI[ B#6, D#0, F#0 ]
  Max compute units:                 60
  Max work items dimensions:             3
    Max work items[0]:                 1024
    Max work items[1]:                 1024
    Max work items[2]:                 1024
  Max work group size:                 256
  Preferred vector width char:             4
  Preferred vector width short:             2
  Preferred vector width int:             1
  Preferred vector width long:             1
  Preferred vector width float:             1
  Preferred vector width double:         1
  Native vector width char:             4
  Native vector width short:             2
  Native vector width int:             1
  Native vector width long:             1
  Native vector width float:             1
  Native vector width double:             1
  Max clock frequency:                 1801Mhz
  Address bits:                     64
  Max memory allocation:             14588628168
  Image support:                 Yes
  Max number of images read arguments:         128
  Max number of images write arguments:         8
  Max image 2D width:                 16384
  Max image 2D height:                 16384
  Max image 3D width:                 16384
  Max image 3D height:                 16384
  Max image 3D depth:                 8192
  Max samplers within kernel:             16
  Max size of kernel argument:             1024
  Alignment (bits) of base address:         1024
  Minimum alignment (bytes) for any datatype:     128
  Single precision floating point capability
    Denorms:                     Yes
    Quiet NaNs:                     Yes
    Round to nearest even:             Yes
    Round to zero:                 Yes
    Round to +ve and infinity:             Yes
    IEEE754-2008 fused multiply-add:         Yes
  Cache type:                     Read/Write
  Cache line size:                 64
  Cache size:                     16384
  Global memory size:                 17163091968
  Constant buffer size:                 14588628168
  Max number of constant args:             8
  Local memory type:                 Local
  Local memory size:                 65536
  Max pipe arguments:                 16
  Max pipe active reservations:             16
  Max pipe packet size:                 1703726280
  Max global variable size:             14588628168
  Max global variable preferred total size:     17163091968
  Max read/write image args:             64
  Max on device events:                 1024
  Queue on device max size:             8388608
  Max on device queues:                 1
  Queue on device preferred size:         262144
  SVM capabilities:                
    Coarse grain buffer:             Yes
    Fine grain buffer:                 Yes
    Fine grain system:                 No
    Atomics:                     No
  Preferred platform atomic alignment:         0
  Preferred global atomic alignment:         0
  Preferred local atomic alignment:         0
  Kernel Preferred work group size multiple:     64
  Error correction support:             0
  Unified memory for Host and Device:         0
  Profiling timer resolution:             1
  Device endianess:                 Little
  Available:                     Yes
  Compiler available:                 Yes
  Execution capabilities:                
    Execute OpenCL kernels:             Yes
    Execute native function:             No
  Queue on Host properties:                
    Out-of-Order:                 No
    Profiling :                     Yes
  Queue on Device properties:                
    Out-of-Order:                 Yes
    Profiling :                     Yes
  Platform ID:                     0x7f57fb5f1808
  Name:                         gfx906:sramecc+:xnack-
  Vendor:                     Advanced Micro Devices, Inc.
  Device OpenCL C version:             OpenCL C 2.0
  Driver version:                 3625.0 (HSA1.1,LC)
  Profile:                     FULL_PROFILE
  Version:                     OpenCL 2.0
  Extensions:                     cl_khr_fp64 cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_3d_image_writes cl_khr_byte_addressable_store cl_khr_fp16 cl_khr_gl_sharing cl_amd_device_attribute_query cl_amd_media_ops cl_amd_media_ops2 cl_khr_image2d_from_buffer cl_khr_subgroups cl_khr_depth_images cl_amd_copy_buffer_p2p cl_amd_assembly_program
reanimator
reanimator
Joined: 14 Feb 23
Posts: 10
Credit: 68570460
RAC: 562124

The NVIDIA build works

The NVIDIA build works perfectly fine: https://einsteinathome.org/task/1688586317

Ian&Steve C.
Ian&Steve C.
Joined: 19 Jan 20
Posts: 4044
Credit: 48018441685
RAC: 35437838

reanimator wrote: What

reanimator wrote:

What drivers do you have installed?

 

modinfo amdgpu

name: amdgpu
vermagic: 6.11.8-300.fc41.x86_64 SMP preempt mod_unload


rocm-clinfo

 

Number of platforms:                 2
  Platform Profile:                 FULL_PROFILE
  Platform Version:                 OpenCL 3.0 CUDA 12.7.33
  Platform Name:                 NVIDIA CUDA
  Platform Vendor:                 NVIDIA Corporation
  Platform Extensions:                 cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_fp64 cl_khr_3d_image_writes cl_khr_byte_addressable_store cl_khr_icd cl_khr_gl_sharing cl_nv_compiler_options cl_nv_device_attribute_query cl_nv_pragma_unroll cl_nv_copy_opts cl_khr_gl_event cl_nv_create_buffer cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_nv_kernel_attribute cl_khr_device_uuid cl_khr_pci_bus_info cl_khr_external_semaphore cl_khr_external_memory cl_khr_external_semaphore_opaque_fd cl_khr_external_memory_opaque_fd cl_khr_semaphore
  Platform Profile:                 FULL_PROFILE
  Platform Version:                 OpenCL 2.1 AMD-APP (3625.0)
  Platform Name:                 AMD Accelerated Parallel Processing
  Platform Vendor:                 Advanced Micro Devices, Inc.
  Platform Extensions:                 cl_khr_icd cl_amd_event_callback
  Platform Name:                 AMD Accelerated Parallel Processing
Number of devices:                 1
  Device Type:                     CL_DEVICE_TYPE_GPU
  Vendor ID:                     1002h
  Board name:                     AMD Radeon VII
  Device Topology:                 PCI[ B#6, D#0, F#0 ]
  Max compute units:                 60
  Max work items dimensions:             3
    Max work items[0]:                 1024
    Max work items[1]:                 1024
    Max work items[2]:                 1024
  Max work group size:                 256
  Preferred vector width char:             4
  Preferred vector width short:             2
  Preferred vector width int:             1
  Preferred vector width long:             1
  Preferred vector width float:             1
  Preferred vector width double:         1
  Native vector width char:             4
  Native vector width short:             2
  Native vector width int:             1
  Native vector width long:             1
  Native vector width float:             1
  Native vector width double:             1
  Max clock frequency:                 1801Mhz
  Address bits:                     64
  Max memory allocation:             14588628168
  Image support:                 Yes
  Max number of images read arguments:         128
  Max number of images write arguments:         8
  Max image 2D width:                 16384
  Max image 2D height:                 16384
  Max image 3D width:                 16384
  Max image 3D height:                 16384
  Max image 3D depth:                 8192
  Max samplers within kernel:             16
  Max size of kernel argument:             1024
  Alignment (bits) of base address:         1024
  Minimum alignment (bytes) for any datatype:     128
  Single precision floating point capability
    Denorms:                     Yes
    Quiet NaNs:                     Yes
    Round to nearest even:             Yes
    Round to zero:                 Yes
    Round to +ve and infinity:             Yes
    IEEE754-2008 fused multiply-add:         Yes
  Cache type:                     Read/Write
  Cache line size:                 64
  Cache size:                     16384
  Global memory size:                 17163091968
  Constant buffer size:                 14588628168
  Max number of constant args:             8
  Local memory type:                 Local
  Local memory size:                 65536
  Max pipe arguments:                 16
  Max pipe active reservations:             16
  Max pipe packet size:                 1703726280
  Max global variable size:             14588628168
  Max global variable preferred total size:     17163091968
  Max read/write image args:             64
  Max on device events:                 1024
  Queue on device max size:             8388608
  Max on device queues:                 1
  Queue on device preferred size:         262144
  SVM capabilities:                
    Coarse grain buffer:             Yes
    Fine grain buffer:                 Yes
    Fine grain system:                 No
    Atomics:                     No
  Preferred platform atomic alignment:         0
  Preferred global atomic alignment:         0
  Preferred local atomic alignment:         0
  Kernel Preferred work group size multiple:     64
  Error correction support:             0
  Unified memory for Host and Device:         0
  Profiling timer resolution:             1
  Device endianess:                 Little
  Available:                     Yes
  Compiler available:                 Yes
  Execution capabilities:                
    Execute OpenCL kernels:             Yes
    Execute native function:             No
  Queue on Host properties:                
    Out-of-Order:                 No
    Profiling :                     Yes
  Queue on Device properties:                
    Out-of-Order:                 Yes
    Profiling :                     Yes
  Platform ID:                     0x7f57fb5f1808
  Name:                         gfx906:sramecc+:xnack-
  Vendor:                     Advanced Micro Devices, Inc.
  Device OpenCL C version:             OpenCL C 2.0
  Driver version:                 3625.0 (HSA1.1,LC)
  Profile:                     FULL_PROFILE
  Version:                     OpenCL 2.0
  Extensions:                     cl_khr_fp64 cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_3d_image_writes cl_khr_byte_addressable_store cl_khr_fp16 cl_khr_gl_sharing cl_amd_device_attribute_query cl_amd_media_ops cl_amd_media_ops2 cl_khr_image2d_from_buffer cl_khr_subgroups cl_khr_depth_images cl_amd_copy_buffer_p2p cl_amd_assembly_program

 

I asked what drivers are installed on the W5700 system producing the errors, this appears to be the clinfo from your Radeon VII/Nvidia system which has not completed any tasks.  

_________________________________________________________________________

San-Fernando-Valley
San-Fernando-Valley
Joined: 16 Mar 16
Posts: 469
Credit: 10397647374
RAC: 4424782

reanimator - just unhide your

reanimator - just unhide your computers.

 

That way you don't have to litter up the forum.

 

cheers

sfv

reanimator
reanimator
Joined: 14 Feb 23
Posts: 10
Credit: 68570460
RAC: 562124

That way you don't have to

That way you don't have to litter up the forum.

The reports should publicly available now.

reanimator
reanimator
Joined: 14 Feb 23
Posts: 10
Credit: 68570460
RAC: 562124

I asked what drivers are

I asked what drivers are installed on the W5700 system producing the errors, this appears to be the clinfo from your Radeon VII/Nvidia system which has not completed any tasks. 

The amdgpu kernel module is use. It is part of signed Linux kernel 6.11.8-300.fc41.x86_64. No extra DKMS driver from ROCm package is installed.

06:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Vega 20 [Radeon VII] (rev c1)
    Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Device 081e
    Kernel driver in use: amdgpu
    Kernel modules: amdgpu

 

@Ian&Steve C. What kind of the information are you looking for?

I guess some LLVM/Clang library is missing that causes ld.lld linker to produce no valid application due to missing function __printf_alloc.

Ian&Steve C.
Ian&Steve C.
Joined: 19 Jan 20
Posts: 4044
Credit: 48018441685
RAC: 35437838

The built in Linux drivers

again you’re showing info from the wrong system. I’m not sure how else to get that across. You have two computers, one with a W5700 and one with a Radeon VII, and both times I’m asking for the drivers installed on the W5700 system and both times you’ve replied with info from the Radeon VII system. 
 

But the built in Linux drivers usually don’t work well on Einstein from what I’ve seen. Try installing the ROCM drivers or the driver installer from AMD’s website for your hardware. 

_________________________________________________________________________

reanimator
reanimator
Joined: 14 Feb 23
Posts: 10
Credit: 68570460
RAC: 562124

again you’re showing info

again you’re showing info from the wrong system. I’m not sure how else to get that across. You have two computers, one with a W5700 and one with a Radeon VII, and both times I’m asking for the drivers installed on the W5700 system and both times you’ve replied with info from the Radeon VII system. 

Sorry for the confusion. Yes, indeed I do have two system with the same FC41 OS installed running the same kernel version. On both systems I observe the same failure with ASGWS tasks. So in respect of installed driver the systems are identical.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.