EasyBuild is a software build and installation framework that allows you to manage (scientific) software on High Performance Computing (HPC) systems in an efficient way.

Overview

https://github.com/easybuilders/easybuild-easyblocks/workflows/easyblocks%20unit%20tests/badge.svg?branch=develop

https://easybuilders.github.io/easybuild/images/easybuild_logo_small.png

EasyBuild is a software build and installation framework that allows you to manage (scientific) software on High Performance Computing (HPC) systems in an efficient way.

The easybuild-easyblocks package provides a collection of easyblocks for EasyBuild. Easyblocks are Python modules that implement the install procedure for a (group of) software package(s). Together with the EasyBuild framework, they allow to easily build and install supported software packages.

The EasyBuild documentation is available at http://easybuild.readthedocs.org/.

The easybuild-easyblocks source code is hosted on GitHub, along with an issue tracker for bug reports and feature requests, see https://github.com/easybuilders/easybuild-easyblocks.

Related Python packages:

Comments
  • Building TensorFlow-2.[23].*-fosscuda-2019b-Python-3.7.4.eb  on Power fails due to lack of `__ieee128`.

    Building TensorFlow-2.[23].*-fosscuda-2019b-Python-3.7.4.eb on Power fails due to lack of `__ieee128`.

    We get:

    INFO: From Compiling tensorflow/core/kernels/bincount_op_gpu.cu.cc:
    /usr/include/bits/floatn.h(79): error: identifier "__ieee128" is undefined
    /usr/include/bits/floatn.h(82): error: invalid argument to attribute "__mode__"
    

    @branfosj and @edmondac have solved this with https://raw.githubusercontent.com/bear-rsg/easybuild-easyconfigs/2019b/easybuild/easyconfigs/t/TensorFlow/TensorFlow-2.0.0_ppc64le_nvcc_options.patch which Ed described as:

    Tell TensorFlow to tell Bazel to tell nvcc to tell G++ to use -mno-float128 and -std=c++11 . One of our "finest". :-)

    The suggestion from @boegel is to do this as an inline patch in the EasyBlock, since -mno-float128 only works on Power... Something ala:

    @local_config_cuda//cuda:using_nvcc": ([ -> "@local_config_cuda//cuda:using_nvcc": ([ "-nvcc_options=compiler-options=-mno-float128", "-nvcc_options=compiler-options=-std=c++11", via apply_regex_substitutions.

    (Also, this probably also applies to PyTorch?)

    Anyway, I will try to look at this soonish. I'd be great to get into 4.3.2.

    problem report 
    opened by terjekv 35
  • generic easyblock for Cray toolchains

    generic easyblock for Cray toolchains

    (requires ~~https://github.com/hpcugent/easybuild-framework/pull/1506~~)

    required for stable Cray support (cfr. https://github.com/hpcugent/easybuild-framework/issues/1390)

    cc @gppezzi

    opened by boegel 31
  • Easyblocks for Anaconda and Conda Env

    Easyblocks for Anaconda and Conda Env

    Here are two easyblocks, referenced here: https://github.com/hpcugent/easybuild-framework/issues/1791

    The first, anaconda.py, installs the python distro anaconda as a module.

    The second, condacreate.py, uses the conda create utility to create a conda environment, and optionally installs requirements into that environment. http://conda.pydata.org/docs/using/envs.html

    Acknowledgements This work is being sponsored by NYU Abu Dhabi Center for Genomics and Systems Biology

    opened by jerowe 29
  • one GAMESS-US easyblock to rule them all (combines #470 and #544) (WIP)

    one GAMESS-US easyblock to rule them all (combines #470 and #544) (WIP)

    This easyblock combines the ones in PR #470 by @pescobar and #544 by @scintimandrion, picking the best of both and combining it in one GAMESS-US easyblock to rule them all.

    Need a bit more test before I consider this final.

    opened by boegel 29
  • update LAMMPS easyblock for LAMMPS/23Jun22

    update LAMMPS easyblock for LAMMPS/23Jun22

    This easyblock supports the patch release 22Oct20 of LAMMPS, which is rc2 for the next stable release. ~~Unfortunately it is not currently backwards compatible so is a WIP until we figure that out.~~

    UPDATE: This now supports the latest stable version of LAMMPS (23Jun2022)

    update 
    opened by ocaisa 27
  • add easyblock for TensorFlow (REVIEW)

    add easyblock for TensorFlow (REVIEW)

    With this easyblock, I'm able to build & install TensorFlow 1.4.0 from source using foss/2017b and Python 3.6.3 (cfr. https://github.com/easybuilders/easybuild-easyconfigs/pull/5318).

    Further testing is required, especially w.r.t. building with Intel compilers on top of Intel MKL & GPU support (i.e., including CUDA & cuDNN as dependencies).

    One thing not taking care of now is avoiding that the installation procedure pulls in a whole bunch of dependencies itself, it seems like Bazel doesn't have good support for providing the dependencies via some other way (meaning that a lot of patching is probably required to avoid that they're automagically installed).

    Also, we can avoid the need for run_cmd_qa by setting $TF_* environment variables that are picked up by configure, it may be worth while switching to that approach.

    new 
    opened by boegel 26
  • Override impi defaults for mpicc etc.

    Override impi defaults for mpicc etc.

    Added a switch so that one can override the defaults of the impi installation. Defaults are to wrap the system compilers with mpicc, mpicxx, mpif90...

    The switch overrides this behaviour by setting environment variables.

    These environment variables also affect the behaviour of mpigcc and mpigxx so I've added aliases to give back the expected behaviour that they wrap the GNU compilers.

    opened by ocaisa 26
  • Modified CUDA easyblock to support wrapper creation

    Modified CUDA easyblock to support wrapper creation

    This PR modifies the CUDA easyblock to support the optional creation of nvcc wrappers. It adds two extra boolean options for easyconfigs: generate_intel_wrapper and generate_gcc_wrapper. The first one will create a wrapper invcc. The second a wrapper called gnvcc.

    This allows to simply tell users "Use invcc to have icpc as a host compiler".

    This is a convenience fix that doesn't modify the default behaviour or installation.

    It will be followed by another PR in the easyconfigs repo, with an example for iccifort+CUDA.

    UPDATE: The naming of the wrappers and how they are created has changed significantly. Take a look at the development of the PR.

    opened by damianam 24
  • generate and install pkg-config files for OpenSSL wrapper

    generate and install pkg-config files for OpenSSL wrapper

    Fixes #2525

    The files for pkg-config have to be specifically generated for the wrapper because the Libs and Cflags found in certain system installations do not apply to the wrapper. For instance, OpenSSL 1.1 in CentOS 7 adds and extra openssl11 subdirectory to some paths.

    I avoided parsing the OpenSSL pkg-config files in the host system because they are not trivial to find:

    • their location is not necessarily relative to the library dir
    • in some distros the path to pkg-config files is not easily predictable as it contains arch dependent subdirectories

    So, this easyblock uses the pkg-config command to extract as much information as possible from the system installation and the rest is hardcoded in the easyblock.

    update: this PR also makes the lookup of system libraries more reliable

    • The wrapper now requires finding all libraries in openssl_libs instead of only a first match.
    • Since system libraries can lack version strings, the wrapper now first checks the filename of the real library for a full version string and only checks the strings within the library as fallback.
    enhancement 
    opened by lexming 23
  • fix installing of Clang with RPATH linking

    fix installing of Clang with RPATH linking

    This is a first attempt at fixing https://github.com/easybuilders/easybuild-easyconfigs/issues/15106

    It requires this small framework change https://github.com/easybuilders/easybuild-framework/pull/4088

    bug fix 
    opened by casparvl 22
  • binutils: zlib not correctly embedded

    binutils: zlib not correctly embedded

    The way in which zlib is embedded in libbfd is half-way broken. While the zlib symbols correctly show up in libbfd.so

    $ nm $EBROOTBINUTILS/lib/libbfd.so | grep inflate
    00000000000f2d80 T inflate
    00000000000f50a0 T inflateCodesUsed
    00000000000f4e00 T inflateCopy
    00000000000f49b0 T inflateEnd
    [...]
    

    they do not in libbfd.a:

    $ nm $EBROOTBINUTILS/lib/libbfd.a | grep inflate
    nm: libz.a: File format not recognized
                     U inflate
                     U inflateEnd
                     U inflateInit_
                     U inflateReset
    

    As can be seen from the first line of the output, the reason is that the whole libz.a is included in libbfd.a rather than the individual object files of libz.a. This leads to problems when linking a program statically.

    Example:

    $ cat foo.c
    #define PACKAGE_NAME
    #define PACKAGE_VERSION
    
    #include <stdio.h>
    
    #include <bfd.h>
    
    int main(int argc, char** argv)
    {
       bfd_init();
    
       bfd * bfdFile = bfd_openr( "/bin/ls", "elf64-x86-64" );
       if ( bfdFile == NULL )
       {
           printf( "Error [%x]: %s\n", bfd_get_error(), bfd_errmsg(bfd_get_error()) );
           return 1;
       }
       if ( !bfd_check_format( bfdFile, bfd_object ))
       {
           printf( "Error [%x]: %s\n", bfd_get_error(), bfd_errmsg(bfd_get_error()) );
           return 1;
       }
    
       bfd_close( bfdFile );
       return 0;
    }
    
    $ gcc -o foo foo.c -static -lbfd -liberty -ldl
    /.../binutils/2.29.1/lib/libbfd.a(plugin.o):plugin.c:function try_load_plugin: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
    /.../binutils/2.29.1/lib/libbfd.a(compress.o):compress.c:function decompress_contents: error: undefined reference to 'inflateInit_'
    /.../binutils/2.29.1/lib/libbfd.a(compress.o):compress.c:function decompress_contents: error: undefined reference to 'inflateReset'
    /.../binutils/2.29.1/lib/libbfd.a(compress.o):compress.c:function decompress_contents: error: undefined reference to 'inflate'
    /.../binutils/2.29.1/lib/libbfd.a(compress.o):compress.c:function decompress_contents: error: undefined reference to 'inflateEnd'
    /.../binutils/2.29.1/lib/libbfd.a(compress.o):compress.c:function bfd_compress_section_contents: error: undefined reference to 'compressBound'
    /.../binutils/2.29.1/lib/libbfd.a(compress.o):compress.c:function bfd_compress_section_contents: error: undefined reference to 'compress'
    

    I'm not entirely sure what the right way to fix this is. Maybe set ZLIB to -L$EBROOTZLIB/lib -lz rather than $EBROOTZLIB/lib/libz.a in all the Makefile.ins?

    problem report 
    opened by geimer 22
  • update scipy easyblock for scipy >= 1.9 to correctly configure BLAS/LAPACK library to use via Meson (WIP)

    update scipy easyblock for scipy >= 1.9 to correctly configure BLAS/LAPACK library to use via Meson (WIP)

    draft PR, since it's not working as intended yet

    The meson setup build command is correct when building on top of foss/2022b:

    The Meson build system
    Version: 0.64.0
    Source dir: /tmp/easybuild_build/SciPybundle/2022.11/gfbf-2022.10/scipy/scipy-1.9.3
    Build dir: /tmp/easybuild_build/SciPybundle/2022.11/gfbf-2022.10/scipy/scipy-1.9.3/build
    Build type: native build
    Project name: SciPy
    Project version: 1.9.3
    C compiler for the host machine: gcc (gcc 12.2.0 "gcc (GCC) 12.2.0")
    C linker for the host machine: gcc ld.bfd 2.39
    ...
    Run-time dependency flexiblas found: YES 3.2.1
    Dependency flexiblas found: YES 3.2.1 (cached)
    

    but the the pip install command also calls meson setup with different arguments, resulting in a direct link with OpenBLAS:

    Using pip 22.3.1 from /software/Python/3.10.8-GCCcore-12.2.0/lib/python3.10/site-packages/pip (python 3.10)
    Processing /tmp/vsc40023/easybuild_build/SciPybundle/2022.11/gfbf-2022.10/scipy/scipy-1.9.3
      Preparing metadata (pyproject.toml): started
      Running command Preparing metadata (pyproject.toml)
      + meson setup --prefix=/software/Python/3.10.8-GCCcore-12.2.0 /tmp/easybuild_build/SciPybundle/2022.11/gfbf-2022.10/scipy/scipy-1.9.3 /tmp/easybuild_build/SciPybundle/2022.11/gfbf-2022.10/scipy/scipy-1.9.3/.mesonpy-znuvtok_/build --native-file=/tmp/easybuild_build/SciPybundle/2022.11/gfbf-2022.10/scipy/scipy-1.9.3/.mesonpy-native-file.ini -Ddebug=false -Doptimization=2
      The Meson build system
      Version: 0.64.0
      Source dir: /tmp/easybuild_build/SciPybundle/2022.11/gfbf-2022.10/scipy/scipy-1.9.3
      Build dir: /tmp/easybuild_build/SciPybundle/2022.11/gfbf-2022.10/scipy/scipy-1.9.3/.mesonpy-znuvtok_/build
      Build type: native build
      Project name: SciPy
      Project version: 1.9.3
      C compiler for the host machine: gcc (gcc 12.2.0 "gcc (GCC) 12.2.0")
      C linker for the host machine: gcc ld.bfd 2.39
      ...
      Run-time dependency openblas found: YES 0.3.21
      Dependency openblas found: YES 0.3.21 (cached)
    

    So rather than running meson setup build before pip install (as suggested by the scipy docs), we probably need to switch to using meson install?

    update 
    opened by boegel 0
  • update RepeatModeler easyblock (repeatmodeler.py) for newer versions 2.0.3 and 2.0.4

    update RepeatModeler easyblock (repeatmodeler.py) for newer versions 2.0.3 and 2.0.4

    Hi EB team,

    Two issues:

    Issue 1: RepeatModeler 2.0.3 and the latest version 2.0.4 changed the configure option from -trf_prgm to -trf_dir . To accommodate this change, a patch file is provided as below: repeatmodeler.py-2.0.3.patch.txt

    Issue 2: Current easyblock complains that the Perl module Devel::Size cannot be found during the RepeatModeler configuration step: perl ./configure [options] This is caused by the fact that the configure script of RepeatModeler 2.0.3 and 2.0.4 now uses Devel::Size. In the current EB recipe (RepeatModeler-2.0.2a-foss-2020b.eb), Devel::Size is installed as a Perl ext, after the configuration step that requires Devel::Size. Solution: Install Deve-Size/0.83 in a separate module and load it as a dependency in EB recipe. Issue 2 will be reported to https://github.com/easybuilders/easybuild-easyconfigs/issues too.

    Thanks so much!

    opened by zhuofeih 0
  • enhance Rosetta easyblock to build with serialization support

    enhance Rosetta easyblock to build with serialization support

    • Fix for database in the version 3.13.
    • Adding the option to compile with mpi,seriallization using a flag in the easyconfig: serialization=True or False
    enhancement 
    opened by alikerr 0
  • enhance `CMakeMake` easyblock to run `ctest` command if `runtest` is `True`

    enhance `CMakeMake` easyblock to run `ctest` command if `runtest` is `True`

    (created using eb --new-pr)

    When using the CMakeMake easyblock or a child of it like CMakeNinja you can use runtest = 'test' to run the tests which is however not user friendly and for CMakeNinja it does not work because it runs make test, so you need test_cmd

    Some EasyBlocks already allow runtest = True to enable running tests, this PR introduces that for CMakeMake and defaults the test_cmd in that case to ctest and (if supported by recent CMake) appends --no-tests=error so that runtest = True will not silently succeed when no tests are found.

    Requires #2837 as setting test_cmd was not enough (error when concatenating the None of runtest) and it needs a default of None for test_cmd so it is able to detect if the user set it or wants the default.

    Written with backward compatibility in mind.

    enhancement 
    opened by Flamefire 2
  • allow use of `test_cmd` without `runtest` for `ConfigureMake`

    allow use of `test_cmd` without `runtest` for `ConfigureMake`

    (created using eb --new-pr)

    When using CMakeNinja and running tests one needs test_cmd = 'ctest' and runtest = '' which is akward but required because it simply concats runtest even when it is not set (None)

    Also handle the default case for test_cmd in the test function so derived classes can detect if the command was set by the user or not. Required for #2838

    And finally clean up the executed command by omitting empty values (e.g. pre/posttestopts not set which resulted in " ctest ")

    enhancement 
    opened by Flamefire 0
Releases(easybuild-easyblocks-v4.6.2)
  • easybuild-easyblocks-v4.6.2(Oct 21, 2022)

  • easybuild-easyblocks-v4.6.1(Sep 12, 2022)

  • easybuild-easyblocks-v4.6.0(Jul 8, 2022)

  • easybuild-easyblocks-v4.5.5(Jun 8, 2022)

  • easybuild-easyblocks-v4.5.4(Mar 31, 2022)

  • easybuild-easyblocks-v4.5.3(Feb 13, 2022)

  • easybuild-easyblocks-v4.5.2(Jan 24, 2022)

  • easybuild-easyblocks-v4.5.1(Dec 13, 2021)

  • easybuild-easyblocks-v4.5.0(Oct 29, 2021)

  • easybuild-easyblocks-v4.4.2(Sep 7, 2021)

  • easybuild-easyblocks-v4.4.1(Jul 6, 2021)

  • easybuild-easyblocks-v4.4.0(Jun 2, 2021)

  • easybuild-easyblocks-v4.3.4(Apr 9, 2021)

  • easybuild-easyblocks-v4.3.3(Feb 23, 2021)

  • easybuild-easyblocks-v4.3.2(Dec 10, 2020)

  • easybuild-easyblocks-v4.3.1(Oct 29, 2020)

  • easybuild-easyblocks-v4.3.0(Sep 13, 2020)

  • easybuild-easyblocks-v4.2.2(Jul 8, 2020)

  • easybuild-easyblocks-v4.2.1(May 20, 2020)

  • easybuild-easyblocks-v4.2.0(Apr 14, 2020)

  • easybuild-easyblocks-v4.1.1(Jan 16, 2020)

A Web app to Cross-Seed torrents in Deluge/qBittorrent/Transmission

SeedCross A Web app to Cross-Seed torrents in Deluge/qBittorrent/Transmission based on CrossSeedAutoDL Require Jackett Deluge/qBittorrent/Transmission

ccf2012 76 Dec 19, 2022
This library attempts to abstract the handling of Sigma rules in Python

This library attempts to abstract the handling of Sigma rules in Python. The rules are parsed using a schema defined with pydantic, and can be easily loaded from YAML files into a structured Python o

Caleb Stewart 44 Oct 29, 2022
Sudoku solver using backtracking

Sudoku solver Sudoku solver using backtracking Basically in sudoku, we want to be able to solve a sudoku puzzle given an input like this, which repres

Kylie 99 Jan 07, 2023
Neogex is a human readable parser standard, being implemented in Python

Neogex (New Expressions) Parsing Standard Much like Regex, Neogex allows for string parsing and validation based on a set of requirements. Unlike Rege

Seamus Donnellan 1 Dec 17, 2021
Wagtail + Lottie is a Wagtail package for playing Adobe After Effects animations exported as json with Bodymovin.

Wagtail Lottie Wagtail + Lottie is a Wagtail package for playing Adobe After Effects animations exported as json with Bodymovin. Usage Export your ani

Alexis Le Baron 7 Aug 18, 2022
Alternative StdLib for Nim for Python targets

Alternative StdLib for Nim for Python targets, hijacks Python StdLib for Nim

Juan Carlos 100 Jan 01, 2023
Check COVID locations of interest against Google location history

Location of Interest Checker Script to compare COVID locations of interest to Google location history. The script produces a map plot (as shown below)

9 Mar 30, 2022
LiteX-Acorn-Baseboard is a baseboard developed around the SQRL's Acorn board (or Nite/LiteFury) expanding their possibilities

LiteX-Acorn-Baseboard is a baseboard developed around the SQRL's Acorn board (or Nite/LiteFury) expanding their possibilities

33 Nov 26, 2022
Dotfiles & list of programs

dotfiles & list of programs So I wanted to just backup my most used files. I have a bad habit, sometimes I get tired of a distro and do a wipe and sta

2 Sep 04, 2022
Scripts to integrate DFIR-IRIS, MISP and TimeSketch

Scripts to integrate DFIR-IRIS, MISP and TimeSketch

Koen Van Impe 20 Dec 16, 2022
This Python script can enumerate all URLs present in robots.txt files, and test whether they can be accessed or not.

Robots.txt tester With this script, you can enumerate all URLs present in robots.txt files, and test whether you can access them or not. Setup Clone t

Podalirius 32 Oct 10, 2022
The official Repository wherein newbies into Open Source can Contribute during the Hacktoberfest 2021

Hacktoberfest 2021 Get Started With your first Contrinution/Pull Request : Fork/Copy the repo by clicking the right most button on top of the page. Go

HacOkars 25 Aug 20, 2022
Check if Python package names are available on PyPI.

😻 isavailable Can I haz this Python package on PyPI? Check if Python package names are available on PyPI. Usage $ isavailable checks whether your des

Felipe S. S. Schneider 3 May 18, 2022
:art: Diagram as Code for prototyping cloud system architectures

Diagrams Diagram as Code. Diagrams lets you draw the cloud system architecture in Python code. It was born for prototyping a new system architecture d

MinJae Kwon 27.5k Jan 04, 2023
synchronize projects via yaml/json manifest. built on libvcs

vcspull - synchronize your repos. built on libvcs Manage your commonly used repos from YAML / JSON manifest(s). Compare to myrepos. Great if you use t

python utilities for version control 200 Dec 20, 2022
Add any Program in any language you like or add a hello world Program ❣️ if you like give us :star:

Welcome to the Hacktoberfest 2018 Hello-world 📋 This Project aims to help you to get started with using Github. You can find a tutorial here What is

Aniket Sharma 1.5k Nov 16, 2022
This python module allows to extract data from the RAW-file-format produces by devices from Thermo Fisher Scientific.

fisher_py This Python module allows access to Thermo Orbitrap raw mass spectrometer files. Using this library makes it possible to automate the analys

8 Oct 14, 2022
A joke conlang with minimal semantics

SyntaxLang Reserved Defined Words Word Function fo Terminates a noun phrase or verb phrase tu Converts an adjective block or sentence to a noun to Ter

Leo Treloar 1 Dec 07, 2021
This suite consists of two different scripts, made to automate attacks against NoSQL databases.

NoSQL-Attack-Suite This suite consists of two different scripts, made to automate attacks against NoSQL databases. The first one looks for a NoSQL Aut

16 Dec 26, 2022
Object-oriented programming (OOP) is a method of structuring a program by bundling related properties and behaviors into individual objects. In this tutorial, you’ll learn the basics of object-oriented programming in Python.

06_Python_Object_Class Introduction 👋 Objected oriented programming as a discipline has gained a universal following among developers. Python, an in-

Milaan Parmar / Милан пармар / _米兰 帕尔马 239 Dec 20, 2022