EasyBuild is a software build and installation framework that allows you to manage (scientific) software on High Performance Computing (HPC) systems in an efficient way.

We get:

INFO: From Compiling tensorflow/core/kernels/bincount_op_gpu.cu.cc:
/usr/include/bits/floatn.h(79): error: identifier "__ieee128" is undefined
/usr/include/bits/floatn.h(82): error: invalid argument to attribute "__mode__"

@branfosj and @edmondac have solved this with https://raw.githubusercontent.com/bear-rsg/easybuild-easyconfigs/2019b/easybuild/easyconfigs/t/TensorFlow/TensorFlow-2.0.0_ppc64le_nvcc_options.patch which Ed described as:

Tell TensorFlow to tell Bazel to tell nvcc to tell G++ to use -mno-float128 and -std=c++11 . One of our "finest". :-)

The suggestion from @boegel is to do this as an inline patch in the EasyBlock, since -mno-float128 only works on Power... Something ala:

@local_config_cuda//cuda:using_nvcc": ([ -> "@local_config_cuda//cuda:using_nvcc": ([ "-nvcc_options=compiler-options=-mno-float128", "-nvcc_options=compiler-options=-std=c++11", via apply_regex_substitutions.

(Also, this probably also applies to PyTorch?)

Anyway, I will try to look at this soonish. I'd be great to get into 4.3.2.

(requires ~~https://github.com/hpcugent/easybuild-framework/pull/1506~~)

required for stable Cray support (cfr. https://github.com/hpcugent/easybuild-framework/issues/1390)

cc @gppezzi

Here are two easyblocks, referenced here: https://github.com/hpcugent/easybuild-framework/issues/1791

The first, anaconda.py, installs the python distro anaconda as a module.

The second, condacreate.py, uses the conda create utility to create a conda environment, and optionally installs requirements into that environment. http://conda.pydata.org/docs/using/envs.html

Acknowledgements This work is being sponsored by NYU Abu Dhabi Center for Genomics and Systems Biology

This easyblock combines the ones in PR #470 by @pescobar and #544 by @scintimandrion, picking the best of both and combining it in one GAMESS-US easyblock to rule them all.

Need a bit more test before I consider this final.

This easyblock supports the patch release 22Oct20 of LAMMPS, which is rc2 for the next stable release. ~~Unfortunately it is not currently backwards compatible so is a WIP until we figure that out.~~

UPDATE: This now supports the latest stable version of LAMMPS (23Jun2022)

With this easyblock, I'm able to build & install TensorFlow 1.4.0 from source using foss/2017b and Python 3.6.3 (cfr. https://github.com/easybuilders/easybuild-easyconfigs/pull/5318).

Further testing is required, especially w.r.t. building with Intel compilers on top of Intel MKL & GPU support (i.e., including CUDA & cuDNN as dependencies).

One thing not taking care of now is avoiding that the installation procedure pulls in a whole bunch of dependencies itself, it seems like Bazel doesn't have good support for providing the dependencies via some other way (meaning that a lot of patching is probably required to avoid that they're automagically installed).

Also, we can avoid the need for run_cmd_qa by setting $TF_* environment variables that are picked up by configure, it may be worth while switching to that approach.

Added a switch so that one can override the defaults of the impi installation. Defaults are to wrap the system compilers with mpicc, mpicxx, mpif90...

The switch overrides this behaviour by setting environment variables.

These environment variables also affect the behaviour of mpigcc and mpigxx so I've added aliases to give back the expected behaviour that they wrap the GNU compilers.

This PR modifies the CUDA easyblock to support the optional creation of nvcc wrappers. It adds two extra boolean options for easyconfigs: generate_intel_wrapper and generate_gcc_wrapper. The first one will create a wrapper invcc. The second a wrapper called gnvcc.

This allows to simply tell users "Use invcc to have icpc as a host compiler".

This is a convenience fix that doesn't modify the default behaviour or installation.

It will be followed by another PR in the easyconfigs repo, with an example for iccifort+CUDA.

UPDATE: The naming of the wrappers and how they are created has changed significantly. Take a look at the development of the PR.

Fixes #2525

The files for pkg-config have to be specifically generated for the wrapper because the Libs and Cflags found in certain system installations do not apply to the wrapper. For instance, OpenSSL 1.1 in CentOS 7 adds and extra openssl11 subdirectory to some paths.

I avoided parsing the OpenSSL pkg-config files in the host system because they are not trivial to find:

• their location is not necessarily relative to the library dir
• in some distros the path to pkg-config files is not easily predictable as it contains arch dependent subdirectories

So, this easyblock uses the pkg-config command to extract as much information as possible from the system installation and the rest is hardcoded in the easyblock.

update: this PR also makes the lookup of system libraries more reliable

• The wrapper now requires finding all libraries in openssl_libs instead of only a first match.
• Since system libraries can lack version strings, the wrapper now first checks the filename of the real library for a full version string and only checks the strings within the library as fallback.

This is a first attempt at fixing https://github.com/easybuilders/easybuild-easyconfigs/issues/15106

It requires this small framework change https://github.com/easybuilders/easybuild-framework/pull/4088

The way in which zlib is embedded in libbfd is half-way broken. While the zlib symbols correctly show up in libbfd.so

$ nm $EBROOTBINUTILS/lib/libbfd.so | grep inflate
00000000000f2d80 T inflate
00000000000f50a0 T inflateCodesUsed
00000000000f4e00 T inflateCopy
00000000000f49b0 T inflateEnd
[...]

they do not in libbfd.a:

$ nm $EBROOTBINUTILS/lib/libbfd.a | grep inflate
nm: libz.a: File format not recognized
                 U inflate
                 U inflateEnd
                 U inflateInit_
                 U inflateReset

As can be seen from the first line of the output, the reason is that the whole libz.a is included in libbfd.a rather than the individual object files of libz.a. This leads to problems when linking a program statically.

Example:

$ cat foo.c
#define PACKAGE_NAME
#define PACKAGE_VERSION

#include <stdio.h>

#include <bfd.h>

int main(int argc, char** argv)
{
   bfd_init();

   bfd * bfdFile = bfd_openr( "/bin/ls", "elf64-x86-64" );
   if ( bfdFile == NULL )
   {
       printf( "Error [%x]: %s\n", bfd_get_error(), bfd_errmsg(bfd_get_error()) );
       return 1;
   }
   if ( !bfd_check_format( bfdFile, bfd_object ))
   {
       printf( "Error [%x]: %s\n", bfd_get_error(), bfd_errmsg(bfd_get_error()) );
       return 1;
   }

   bfd_close( bfdFile );
   return 0;
}

$ gcc -o foo foo.c -static -lbfd -liberty -ldl
/.../binutils/2.29.1/lib/libbfd.a(plugin.o):plugin.c:function try_load_plugin: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/.../binutils/2.29.1/lib/libbfd.a(compress.o):compress.c:function decompress_contents: error: undefined reference to 'inflateInit_'
/.../binutils/2.29.1/lib/libbfd.a(compress.o):compress.c:function decompress_contents: error: undefined reference to 'inflateReset'
/.../binutils/2.29.1/lib/libbfd.a(compress.o):compress.c:function decompress_contents: error: undefined reference to 'inflate'
/.../binutils/2.29.1/lib/libbfd.a(compress.o):compress.c:function decompress_contents: error: undefined reference to 'inflateEnd'
/.../binutils/2.29.1/lib/libbfd.a(compress.o):compress.c:function bfd_compress_section_contents: error: undefined reference to 'compressBound'
/.../binutils/2.29.1/lib/libbfd.a(compress.o):compress.c:function bfd_compress_section_contents: error: undefined reference to 'compress'

I'm not entirely sure what the right way to fix this is. Maybe set ZLIB to -L$EBROOTZLIB/lib -lz rather than $EBROOTZLIB/lib/libz.a in all the Makefile.ins?

draft PR, since it's not working as intended yet

The meson setup build command is correct when building on top of foss/2022b:

The Meson build system
Version: 0.64.0
Source dir: /tmp/easybuild_build/SciPybundle/2022.11/gfbf-2022.10/scipy/scipy-1.9.3
Build dir: /tmp/easybuild_build/SciPybundle/2022.11/gfbf-2022.10/scipy/scipy-1.9.3/build
Build type: native build
Project name: SciPy
Project version: 1.9.3
C compiler for the host machine: gcc (gcc 12.2.0 "gcc (GCC) 12.2.0")
C linker for the host machine: gcc ld.bfd 2.39
...
Run-time dependency flexiblas found: YES 3.2.1
Dependency flexiblas found: YES 3.2.1 (cached)

but the the pip install command also calls meson setup with different arguments, resulting in a direct link with OpenBLAS:

Using pip 22.3.1 from /software/Python/3.10.8-GCCcore-12.2.0/lib/python3.10/site-packages/pip (python 3.10)
Processing /tmp/vsc40023/easybuild_build/SciPybundle/2022.11/gfbf-2022.10/scipy/scipy-1.9.3
  Preparing metadata (pyproject.toml): started
  Running command Preparing metadata (pyproject.toml)
  + meson setup --prefix=/software/Python/3.10.8-GCCcore-12.2.0 /tmp/easybuild_build/SciPybundle/2022.11/gfbf-2022.10/scipy/scipy-1.9.3 /tmp/easybuild_build/SciPybundle/2022.11/gfbf-2022.10/scipy/scipy-1.9.3/.mesonpy-znuvtok_/build --native-file=/tmp/easybuild_build/SciPybundle/2022.11/gfbf-2022.10/scipy/scipy-1.9.3/.mesonpy-native-file.ini -Ddebug=false -Doptimization=2
  The Meson build system
  Version: 0.64.0
  Source dir: /tmp/easybuild_build/SciPybundle/2022.11/gfbf-2022.10/scipy/scipy-1.9.3
  Build dir: /tmp/easybuild_build/SciPybundle/2022.11/gfbf-2022.10/scipy/scipy-1.9.3/.mesonpy-znuvtok_/build
  Build type: native build
  Project name: SciPy
  Project version: 1.9.3
  C compiler for the host machine: gcc (gcc 12.2.0 "gcc (GCC) 12.2.0")
  C linker for the host machine: gcc ld.bfd 2.39
  ...
  Run-time dependency openblas found: YES 0.3.21
  Dependency openblas found: YES 0.3.21 (cached)

So rather than running meson setup build before pip install (as suggested by the scipy docs), we probably need to switch to using meson install?

Hi EB team,

Two issues:

Issue 1: RepeatModeler 2.0.3 and the latest version 2.0.4 changed the configure option from -trf_prgm to -trf_dir . To accommodate this change, a patch file is provided as below: repeatmodeler.py-2.0.3.patch.txt

Issue 2: Current easyblock complains that the Perl module Devel::Size cannot be found during the RepeatModeler configuration step: perl ./configure [options] This is caused by the fact that the configure script of RepeatModeler 2.0.3 and 2.0.4 now uses Devel::Size. In the current EB recipe (RepeatModeler-2.0.2a-foss-2020b.eb), Devel::Size is installed as a Perl ext, after the configuration step that requires Devel::Size. Solution: Install Deve-Size/0.83 in a separate module and load it as a dependency in EB recipe. Issue 2 will be reported to https://github.com/easybuilders/easybuild-easyconfigs/issues too.

Thanks so much!

• Fix for database in the version 3.13.
• Adding the option to compile with mpi,seriallization using a flag in the easyconfig: serialization=True or False

(created using eb --new-pr)

When using the CMakeMake easyblock or a child of it like CMakeNinja you can use runtest = 'test' to run the tests which is however not user friendly and for CMakeNinja it does not work because it runs make test, so you need test_cmd

Some EasyBlocks already allow runtest = True to enable running tests, this PR introduces that for CMakeMake and defaults the test_cmd in that case to ctest and (if supported by recent CMake) appends --no-tests=error so that runtest = True will not silently succeed when no tests are found.

Requires #2837 as setting test_cmd was not enough (error when concatenating the None of runtest) and it needs a default of None for test_cmd so it is able to detect if the user set it or wants the default.

Written with backward compatibility in mind.

(created using eb --new-pr)

When using CMakeNinja and running tests one needs test_cmd = 'ctest' and runtest = '' which is akward but required because it simply concats runtest even when it is not set (None)

Also handle the default case for test_cmd in the test function so derived classes can detect if the command was set by the user or not. Required for #2838

And finally clean up the executed command by omitting empty values (e.g. pre/posttestopts not set which resulted in " ctest ")