EasyBuild is a software build and installation framework that allows you to manage (scientific) software on High Performance Computing (HPC) systems in an efficient way.

Overview

https://github.com/easybuilders/easybuild-easyblocks/workflows/easyblocks%20unit%20tests/badge.svg?branch=develop

https://easybuilders.github.io/easybuild/images/easybuild_logo_small.png

EasyBuild is a software build and installation framework that allows you to manage (scientific) software on High Performance Computing (HPC) systems in an efficient way.

The easybuild-easyblocks package provides a collection of easyblocks for EasyBuild. Easyblocks are Python modules that implement the install procedure for a (group of) software package(s). Together with the EasyBuild framework, they allow to easily build and install supported software packages.

The EasyBuild documentation is available at http://easybuild.readthedocs.org/.

The easybuild-easyblocks source code is hosted on GitHub, along with an issue tracker for bug reports and feature requests, see https://github.com/easybuilders/easybuild-easyblocks.

Related Python packages:

Comments
  • Building TensorFlow-2.[23].*-fosscuda-2019b-Python-3.7.4.eb  on Power fails due to lack of `__ieee128`.

    Building TensorFlow-2.[23].*-fosscuda-2019b-Python-3.7.4.eb on Power fails due to lack of `__ieee128`.

    We get:

    INFO: From Compiling tensorflow/core/kernels/bincount_op_gpu.cu.cc:
    /usr/include/bits/floatn.h(79): error: identifier "__ieee128" is undefined
    /usr/include/bits/floatn.h(82): error: invalid argument to attribute "__mode__"
    

    @branfosj and @edmondac have solved this with https://raw.githubusercontent.com/bear-rsg/easybuild-easyconfigs/2019b/easybuild/easyconfigs/t/TensorFlow/TensorFlow-2.0.0_ppc64le_nvcc_options.patch which Ed described as:

    Tell TensorFlow to tell Bazel to tell nvcc to tell G++ to use -mno-float128 and -std=c++11 . One of our "finest". :-)

    The suggestion from @boegel is to do this as an inline patch in the EasyBlock, since -mno-float128 only works on Power... Something ala:

    @local_config_cuda//cuda:using_nvcc": ([ -> "@local_config_cuda//cuda:using_nvcc": ([ "-nvcc_options=compiler-options=-mno-float128", "-nvcc_options=compiler-options=-std=c++11", via apply_regex_substitutions.

    (Also, this probably also applies to PyTorch?)

    Anyway, I will try to look at this soonish. I'd be great to get into 4.3.2.

    problem report 
    opened by terjekv 35
  • generic easyblock for Cray toolchains

    generic easyblock for Cray toolchains

    (requires ~~https://github.com/hpcugent/easybuild-framework/pull/1506~~)

    required for stable Cray support (cfr. https://github.com/hpcugent/easybuild-framework/issues/1390)

    cc @gppezzi

    opened by boegel 31
  • Easyblocks for Anaconda and Conda Env

    Easyblocks for Anaconda and Conda Env

    Here are two easyblocks, referenced here: https://github.com/hpcugent/easybuild-framework/issues/1791

    The first, anaconda.py, installs the python distro anaconda as a module.

    The second, condacreate.py, uses the conda create utility to create a conda environment, and optionally installs requirements into that environment. http://conda.pydata.org/docs/using/envs.html

    Acknowledgements This work is being sponsored by NYU Abu Dhabi Center for Genomics and Systems Biology

    opened by jerowe 29
  • one GAMESS-US easyblock to rule them all (combines #470 and #544) (WIP)

    one GAMESS-US easyblock to rule them all (combines #470 and #544) (WIP)

    This easyblock combines the ones in PR #470 by @pescobar and #544 by @scintimandrion, picking the best of both and combining it in one GAMESS-US easyblock to rule them all.

    Need a bit more test before I consider this final.

    opened by boegel 29
  • update LAMMPS easyblock for LAMMPS/23Jun22

    update LAMMPS easyblock for LAMMPS/23Jun22

    This easyblock supports the patch release 22Oct20 of LAMMPS, which is rc2 for the next stable release. ~~Unfortunately it is not currently backwards compatible so is a WIP until we figure that out.~~

    UPDATE: This now supports the latest stable version of LAMMPS (23Jun2022)

    update 
    opened by ocaisa 27
  • add easyblock for TensorFlow (REVIEW)

    add easyblock for TensorFlow (REVIEW)

    With this easyblock, I'm able to build & install TensorFlow 1.4.0 from source using foss/2017b and Python 3.6.3 (cfr. https://github.com/easybuilders/easybuild-easyconfigs/pull/5318).

    Further testing is required, especially w.r.t. building with Intel compilers on top of Intel MKL & GPU support (i.e., including CUDA & cuDNN as dependencies).

    One thing not taking care of now is avoiding that the installation procedure pulls in a whole bunch of dependencies itself, it seems like Bazel doesn't have good support for providing the dependencies via some other way (meaning that a lot of patching is probably required to avoid that they're automagically installed).

    Also, we can avoid the need for run_cmd_qa by setting $TF_* environment variables that are picked up by configure, it may be worth while switching to that approach.

    new 
    opened by boegel 26
  • Override impi defaults for mpicc etc.

    Override impi defaults for mpicc etc.

    Added a switch so that one can override the defaults of the impi installation. Defaults are to wrap the system compilers with mpicc, mpicxx, mpif90...

    The switch overrides this behaviour by setting environment variables.

    These environment variables also affect the behaviour of mpigcc and mpigxx so I've added aliases to give back the expected behaviour that they wrap the GNU compilers.

    opened by ocaisa 26
  • Modified CUDA easyblock to support wrapper creation

    Modified CUDA easyblock to support wrapper creation

    This PR modifies the CUDA easyblock to support the optional creation of nvcc wrappers. It adds two extra boolean options for easyconfigs: generate_intel_wrapper and generate_gcc_wrapper. The first one will create a wrapper invcc. The second a wrapper called gnvcc.

    This allows to simply tell users "Use invcc to have icpc as a host compiler".

    This is a convenience fix that doesn't modify the default behaviour or installation.

    It will be followed by another PR in the easyconfigs repo, with an example for iccifort+CUDA.

    UPDATE: The naming of the wrappers and how they are created has changed significantly. Take a look at the development of the PR.

    opened by damianam 24
  • generate and install pkg-config files for OpenSSL wrapper

    generate and install pkg-config files for OpenSSL wrapper

    Fixes #2525

    The files for pkg-config have to be specifically generated for the wrapper because the Libs and Cflags found in certain system installations do not apply to the wrapper. For instance, OpenSSL 1.1 in CentOS 7 adds and extra openssl11 subdirectory to some paths.

    I avoided parsing the OpenSSL pkg-config files in the host system because they are not trivial to find:

    • their location is not necessarily relative to the library dir
    • in some distros the path to pkg-config files is not easily predictable as it contains arch dependent subdirectories

    So, this easyblock uses the pkg-config command to extract as much information as possible from the system installation and the rest is hardcoded in the easyblock.

    update: this PR also makes the lookup of system libraries more reliable

    • The wrapper now requires finding all libraries in openssl_libs instead of only a first match.
    • Since system libraries can lack version strings, the wrapper now first checks the filename of the real library for a full version string and only checks the strings within the library as fallback.
    enhancement 
    opened by lexming 23
  • fix installing of Clang with RPATH linking

    fix installing of Clang with RPATH linking

    This is a first attempt at fixing https://github.com/easybuilders/easybuild-easyconfigs/issues/15106

    It requires this small framework change https://github.com/easybuilders/easybuild-framework/pull/4088

    bug fix 
    opened by casparvl 22
  • binutils: zlib not correctly embedded

    binutils: zlib not correctly embedded

    The way in which zlib is embedded in libbfd is half-way broken. While the zlib symbols correctly show up in libbfd.so

    $ nm $EBROOTBINUTILS/lib/libbfd.so | grep inflate
    00000000000f2d80 T inflate
    00000000000f50a0 T inflateCodesUsed
    00000000000f4e00 T inflateCopy
    00000000000f49b0 T inflateEnd
    [...]
    

    they do not in libbfd.a:

    $ nm $EBROOTBINUTILS/lib/libbfd.a | grep inflate
    nm: libz.a: File format not recognized
                     U inflate
                     U inflateEnd
                     U inflateInit_
                     U inflateReset
    

    As can be seen from the first line of the output, the reason is that the whole libz.a is included in libbfd.a rather than the individual object files of libz.a. This leads to problems when linking a program statically.

    Example:

    $ cat foo.c
    #define PACKAGE_NAME
    #define PACKAGE_VERSION
    
    #include <stdio.h>
    
    #include <bfd.h>
    
    int main(int argc, char** argv)
    {
       bfd_init();
    
       bfd * bfdFile = bfd_openr( "/bin/ls", "elf64-x86-64" );
       if ( bfdFile == NULL )
       {
           printf( "Error [%x]: %s\n", bfd_get_error(), bfd_errmsg(bfd_get_error()) );
           return 1;
       }
       if ( !bfd_check_format( bfdFile, bfd_object ))
       {
           printf( "Error [%x]: %s\n", bfd_get_error(), bfd_errmsg(bfd_get_error()) );
           return 1;
       }
    
       bfd_close( bfdFile );
       return 0;
    }
    
    $ gcc -o foo foo.c -static -lbfd -liberty -ldl
    /.../binutils/2.29.1/lib/libbfd.a(plugin.o):plugin.c:function try_load_plugin: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
    /.../binutils/2.29.1/lib/libbfd.a(compress.o):compress.c:function decompress_contents: error: undefined reference to 'inflateInit_'
    /.../binutils/2.29.1/lib/libbfd.a(compress.o):compress.c:function decompress_contents: error: undefined reference to 'inflateReset'
    /.../binutils/2.29.1/lib/libbfd.a(compress.o):compress.c:function decompress_contents: error: undefined reference to 'inflate'
    /.../binutils/2.29.1/lib/libbfd.a(compress.o):compress.c:function decompress_contents: error: undefined reference to 'inflateEnd'
    /.../binutils/2.29.1/lib/libbfd.a(compress.o):compress.c:function bfd_compress_section_contents: error: undefined reference to 'compressBound'
    /.../binutils/2.29.1/lib/libbfd.a(compress.o):compress.c:function bfd_compress_section_contents: error: undefined reference to 'compress'
    

    I'm not entirely sure what the right way to fix this is. Maybe set ZLIB to -L$EBROOTZLIB/lib -lz rather than $EBROOTZLIB/lib/libz.a in all the Makefile.ins?

    problem report 
    opened by geimer 22
  • update scipy easyblock for scipy >= 1.9 to correctly configure BLAS/LAPACK library to use via Meson (WIP)

    update scipy easyblock for scipy >= 1.9 to correctly configure BLAS/LAPACK library to use via Meson (WIP)

    draft PR, since it's not working as intended yet

    The meson setup build command is correct when building on top of foss/2022b:

    The Meson build system
    Version: 0.64.0
    Source dir: /tmp/easybuild_build/SciPybundle/2022.11/gfbf-2022.10/scipy/scipy-1.9.3
    Build dir: /tmp/easybuild_build/SciPybundle/2022.11/gfbf-2022.10/scipy/scipy-1.9.3/build
    Build type: native build
    Project name: SciPy
    Project version: 1.9.3
    C compiler for the host machine: gcc (gcc 12.2.0 "gcc (GCC) 12.2.0")
    C linker for the host machine: gcc ld.bfd 2.39
    ...
    Run-time dependency flexiblas found: YES 3.2.1
    Dependency flexiblas found: YES 3.2.1 (cached)
    

    but the the pip install command also calls meson setup with different arguments, resulting in a direct link with OpenBLAS:

    Using pip 22.3.1 from /software/Python/3.10.8-GCCcore-12.2.0/lib/python3.10/site-packages/pip (python 3.10)
    Processing /tmp/vsc40023/easybuild_build/SciPybundle/2022.11/gfbf-2022.10/scipy/scipy-1.9.3
      Preparing metadata (pyproject.toml): started
      Running command Preparing metadata (pyproject.toml)
      + meson setup --prefix=/software/Python/3.10.8-GCCcore-12.2.0 /tmp/easybuild_build/SciPybundle/2022.11/gfbf-2022.10/scipy/scipy-1.9.3 /tmp/easybuild_build/SciPybundle/2022.11/gfbf-2022.10/scipy/scipy-1.9.3/.mesonpy-znuvtok_/build --native-file=/tmp/easybuild_build/SciPybundle/2022.11/gfbf-2022.10/scipy/scipy-1.9.3/.mesonpy-native-file.ini -Ddebug=false -Doptimization=2
      The Meson build system
      Version: 0.64.0
      Source dir: /tmp/easybuild_build/SciPybundle/2022.11/gfbf-2022.10/scipy/scipy-1.9.3
      Build dir: /tmp/easybuild_build/SciPybundle/2022.11/gfbf-2022.10/scipy/scipy-1.9.3/.mesonpy-znuvtok_/build
      Build type: native build
      Project name: SciPy
      Project version: 1.9.3
      C compiler for the host machine: gcc (gcc 12.2.0 "gcc (GCC) 12.2.0")
      C linker for the host machine: gcc ld.bfd 2.39
      ...
      Run-time dependency openblas found: YES 0.3.21
      Dependency openblas found: YES 0.3.21 (cached)
    

    So rather than running meson setup build before pip install (as suggested by the scipy docs), we probably need to switch to using meson install?

    update 
    opened by boegel 0
  • update RepeatModeler easyblock (repeatmodeler.py) for newer versions 2.0.3 and 2.0.4

    update RepeatModeler easyblock (repeatmodeler.py) for newer versions 2.0.3 and 2.0.4

    Hi EB team,

    Two issues:

    Issue 1: RepeatModeler 2.0.3 and the latest version 2.0.4 changed the configure option from -trf_prgm to -trf_dir . To accommodate this change, a patch file is provided as below: repeatmodeler.py-2.0.3.patch.txt

    Issue 2: Current easyblock complains that the Perl module Devel::Size cannot be found during the RepeatModeler configuration step: perl ./configure [options] This is caused by the fact that the configure script of RepeatModeler 2.0.3 and 2.0.4 now uses Devel::Size. In the current EB recipe (RepeatModeler-2.0.2a-foss-2020b.eb), Devel::Size is installed as a Perl ext, after the configuration step that requires Devel::Size. Solution: Install Deve-Size/0.83 in a separate module and load it as a dependency in EB recipe. Issue 2 will be reported to https://github.com/easybuilders/easybuild-easyconfigs/issues too.

    Thanks so much!

    opened by zhuofeih 0
  • enhance Rosetta easyblock to build with serialization support

    enhance Rosetta easyblock to build with serialization support

    • Fix for database in the version 3.13.
    • Adding the option to compile with mpi,seriallization using a flag in the easyconfig: serialization=True or False
    enhancement 
    opened by alikerr 0
  • enhance `CMakeMake` easyblock to run `ctest` command if `runtest` is `True`

    enhance `CMakeMake` easyblock to run `ctest` command if `runtest` is `True`

    (created using eb --new-pr)

    When using the CMakeMake easyblock or a child of it like CMakeNinja you can use runtest = 'test' to run the tests which is however not user friendly and for CMakeNinja it does not work because it runs make test, so you need test_cmd

    Some EasyBlocks already allow runtest = True to enable running tests, this PR introduces that for CMakeMake and defaults the test_cmd in that case to ctest and (if supported by recent CMake) appends --no-tests=error so that runtest = True will not silently succeed when no tests are found.

    Requires #2837 as setting test_cmd was not enough (error when concatenating the None of runtest) and it needs a default of None for test_cmd so it is able to detect if the user set it or wants the default.

    Written with backward compatibility in mind.

    enhancement 
    opened by Flamefire 2
  • allow use of `test_cmd` without `runtest` for `ConfigureMake`

    allow use of `test_cmd` without `runtest` for `ConfigureMake`

    (created using eb --new-pr)

    When using CMakeNinja and running tests one needs test_cmd = 'ctest' and runtest = '' which is akward but required because it simply concats runtest even when it is not set (None)

    Also handle the default case for test_cmd in the test function so derived classes can detect if the command was set by the user or not. Required for #2838

    And finally clean up the executed command by omitting empty values (e.g. pre/posttestopts not set which resulted in " ctest ")

    enhancement 
    opened by Flamefire 0
Releases(easybuild-easyblocks-v4.6.2)
  • easybuild-easyblocks-v4.6.2(Oct 21, 2022)

  • easybuild-easyblocks-v4.6.1(Sep 12, 2022)

  • easybuild-easyblocks-v4.6.0(Jul 8, 2022)

  • easybuild-easyblocks-v4.5.5(Jun 8, 2022)

  • easybuild-easyblocks-v4.5.4(Mar 31, 2022)

  • easybuild-easyblocks-v4.5.3(Feb 13, 2022)

  • easybuild-easyblocks-v4.5.2(Jan 24, 2022)

  • easybuild-easyblocks-v4.5.1(Dec 13, 2021)

  • easybuild-easyblocks-v4.5.0(Oct 29, 2021)

  • easybuild-easyblocks-v4.4.2(Sep 7, 2021)

  • easybuild-easyblocks-v4.4.1(Jul 6, 2021)

  • easybuild-easyblocks-v4.4.0(Jun 2, 2021)

  • easybuild-easyblocks-v4.3.4(Apr 9, 2021)

  • easybuild-easyblocks-v4.3.3(Feb 23, 2021)

  • easybuild-easyblocks-v4.3.2(Dec 10, 2020)

  • easybuild-easyblocks-v4.3.1(Oct 29, 2020)

  • easybuild-easyblocks-v4.3.0(Sep 13, 2020)

  • easybuild-easyblocks-v4.2.2(Jul 8, 2020)

  • easybuild-easyblocks-v4.2.1(May 20, 2020)

  • easybuild-easyblocks-v4.2.0(Apr 14, 2020)

  • easybuild-easyblocks-v4.1.1(Jan 16, 2020)

A small site to list shared directories

Nebula Server Directories This site can be used to list folder and subdirectories in your server : Python It's required to have Python 3.8 or more ins

Adrien J. 1 Dec 28, 2021
KeyBrowser: A program launches a browser and a keylogger at the same time, is used to retrieve a person's personal information

KeyBrowser: A program launches a browser and a keylogger at the same time, is used to retrieve a person's personal information

3 Oct 16, 2022
MuMMI Core is the underlying infrastructure and generalizable component of the MuMMI framework

MuMMI Core is the underlying infrastructure and generalizable component of the MuMMI framework, which facilitates the coordination of massively parallel multiscale simulations.

4 Aug 17, 2022
Utility functions for working with data from Nix in Python

Pynixutil - Utility functions for working with data from Nix in Python Examples Base32 encoding/decoding import pynixutil input = "v5sv61sszx301i0x6x

Tweag 11 Dec 16, 2022
An app to help people apply for admissions on schools/hostels

Admission-helper About An app to help people apply for admissions on schools/hostels This app is a rewrite of Admission-helper-beta-v5.8.9 and I impor

Advik 3 Apr 24, 2022
Model Quantization Benchmark

MQBench Update V0.0.2 Fix academic prepare setting. More deployable prepare process. Fix setup.py. Fix deploy on SNPE. Fix convert_deploy bug. Add Qua

500 Jan 06, 2023
Kunai Shitty Raider Leaked LMFAO

Kunai-Raider-Leaked Kunai Shitty Raider Leaked LMFA

5 Nov 24, 2021
A multi-platform fuzzer for poking at userland binaries and servers

litefuzz A multi-platform fuzzer for poking at userland binaries and servers litefuzz intro why how it works what it does what it doesn't do support p

52 Nov 18, 2022
Today I Commit (1일 1커밋) 챌린지 알림 봇

Today I Commit Challenge 1일1커밋 챌린지를 위한 알림 봇 config.py github_token = "github private access key" slack_token = "slack authorization token" channel = "

sunho 4 Nov 08, 2021
Мой первый калькулятор!!!!!!

my_first_calculator Первый калькулятор созданный мною на питоне Версия калькулятора: 0.0.4 Как скачать? TERMUX Для скрипта нужен питон, скачиваем pkg

Lesha Russkiyov 2 Dec 29, 2021
A check numbers python module

Made with Python3 (C) @FayasNoushad Copyright permission under MIT License License - https://github.com/FayasNoushad/Numbers/blob/main/LICENSE Deplo

Fayas Noushad 3 Nov 28, 2021
Ferramenta de monitoramento do risco de colapso no sistema de saúde em municípios brasileiros com a Covid-19.

FarolCovid 🚦 Ferramenta de monitoramento do risco de colapso no sistema de saúde em municípios brasileiros com a Covid-19. Monitoring tool & simulati

Impulso 49 Jul 10, 2022
A fancy and practical functional tools

Funcy A collection of fancy functional tools focused on practicality. Inspired by clojure, underscore and my own abstractions. Keep reading to get an

Alexander Schepanovski 2.9k Dec 29, 2022
A light library to build tiny websites

A light library to build tiny websites

BT.Q 1 Dec 23, 2021
Xoroshiro-cairo - A xoroshiro128** pseudorandom number generator implementation in Cairo

xoroshiro-cairo A xoroshiro128** pseudorandom number generator implementation in

Milan Cermak 26 Oct 05, 2022
Public Management System for ACP's 24H TT Fronteira 2021

CROWD MANAGEMENT SYSTEM 24H TT Vila de Froteira 2021 This python script creates a dashboard with realtime updates regarding the capacity of spectactor

VOST Portugal 1 Nov 24, 2021
Syntax highlighting for yarn.lock and bun.lockb files

Yarn.lock Syntax Highlighting Syntax highlighting for yarn.lock and bun.lockb files Installation Plugin is not publushed yet on Package Control, to in

Alexander Kuznetsov 4 Jul 06, 2022
Free version of Okuru selfbot, okuru.xyz

Indigo Selfbot Free OpenSource selfbot, Premium version can be found at https://okuru.xyz (5$.) Usage python[3] main.py Installation To install you ca

Dimitri Demarkus 31 Aug 07, 2022
Collie is for uncovering RDMA NIC performance anomalies

Collie is for uncovering RDMA NIC performance anomalies. Overview Prerequ

Bytedance Inc. 34 Dec 11, 2022
A project for the Qvault Hackathon, 2022-01-17

musical-octo-engine Steps to run brew install python-tk brew install portaudio

Erik Kristofer Anderson 2 May 17, 2022