The Multi-Mission Maximum Likelihood framework (3ML)

Overview

CI Conda Build and Publish Test Against XSPEC codecov Documentation Status License

GitHub pull requests GitHub issues

PyPi

PyPI version fury.io PyPI - Downloads PyPI - Python Version

Conda

Conda Conda

drawing

The Multi-Mission Maximum Likelihood framework (3ML)

A framework for multi-wavelength/multi-messenger analysis for astronomy/astrophysics.


Astrophysical sources are observed by different instruments at different wavelengths with an unprecedented quality. Putting all these data together to form a coherent view, however, is a very difficult task. Indeed, each instrument and data type has its own ad-hoc software and handling procedure, which present steep learning curves and do not talk to each other.

The Multi-Mission Maximum Likelihood framework (3ML) provides a common high-level interface and model definition, which allows for an easy, coherent and intuitive modeling of sources using all the available data, no matter their origin. At the same time, thanks to its architecture based on plug-ins, 3ML uses under the hood the official software of each instrument, the only one certified and maintained by the collaboration which built the instrument itself. This guarantees that 3ML is always using the best possible methodology to deal with the data of each instrument.

drawing

Though Maximum Likelihood is in the name for historical reasons, 3ML is an interface to several Bayesian inference algorithms such as MCMC and nested sampling as well as likelihood optimization algorithms. Each approach to analysis can be seamlessly switched between allowing users to try different approaches quickly and without having to rewrite their model or data interfaces.

Like your XPSEC models? You can use them in 3ML as well as our growing selection of 1-,2- and 3-D models from our fast and customizable modeling language astromodels.

Installation

Installing with pip or conda is easy

pip install astromodels threeml
conda  install astromodels threeml -c threeml conda-forge 

Please refer to the Installation instructions for more details and trouble-shooting.

Press

Who is using 3ML?

Here is a highlight list of teams and their publications using 3ML.

A full list of publications using 3ML is here.

Citing

If you find this package useful in you analysis, or the code in your own custom data tools, please cite:

Vianello et al. (2015)

Acknowledgements

3ML makes use of the Spanish Virtual Observatory's Filter Profile service (http://svo2.cab.inta-csic.es/svo/theory/fps3/index.php?mode=browse&gname=NIRT).

If you use these profiles in your research, please consider citing them by using the following suggested sentence in your paper:

"This research has made use of the SVO Filter Profile Service (http://svo2.cab.inta-csic.es/theory/fps/) supported from the Spanish MINECO through grant AyA2014-55216"

and citing the following publications:

The SVO Filter Profile Service. Rodrigo, C., Solano, E., Bayo, A. http://ivoa.net/documents/Notes/SVOFPS/index.html The Filter Profile Service Access Protocol. Rodrigo, C., Solano, E. http://ivoa.net/documents/Notes/SVOFPSDAL/index.html

ThreeML is supported by National Science Foundation (NSF)

Comments
  • Use from astromodels Xspec import * error

    Use from astromodels Xspec import * error

    Hello! When I want to use xspec library to combine GBM and XRT to analyze energy spectrum, I try to use from astromodels Xspec import *has the following error messages. How did this happen?


    ImportError Traceback (most recent call last) /tmp/ipykernel_12482/3995023305.py in 5 6 # we will need XPSEC models for extinction ----> 7 from astromodels.xspec import * 8 from astromodels.xspec.xspec_settings import *

    ~/miniconda3/envs/threeML/lib/python3.7/site-packages/astromodels/xspec/init.py in ----> 1 from .factory import * 2 #from .xspec_settings import x

    ~/miniconda3/envs/threeML/lib/python3.7/site-packages/astromodels/xspec/factory.py in 16 pass 17 ---> 18 from astromodels.xspec import _xspec 19 20 # When running in a Anaconda environment, the package xspec-modelsonly

    ImportError: cannot import name '_xspec' from 'astromodels.xspec' (/home/hank/miniconda3/envs/threeML/lib/python3.7/site-packages/astromodels/xspec/init.py)

    opened by ghost 64
  • How do I install a stable threeml module?

    How do I install a stable threeml module?

    Hello,

    The threeml module is currently latest, causing many modules to fail to install, including the Xspexcmodule. So I want to install a more stable threeml module, so how do I do that?

    opened by ghost 18
  • Background Modeling

    Background Modeling

    I've been looking at the background modeling and it may be better to approach it by adding a node onto PointSource:

    ps = PointSource(...,background_shape=Powerlaw() )
    

    Otherwise, we are just adding a function inside of the plugin and have to dig into private members to access the free parameters of that function.

    Is this too involved of an approach @giacomov ?

    opened by grburgess 18
  • properties of the spectra and choosing sources for SpectrumLike

    properties of the spectra and choosing sources for SpectrumLike

    would it make sense to assign some properties in the SpectrumLike plugin. it is quite useful when one wants to associate only some sources to a given spectrum, instead of summing all, as it is done now.

    opened by volodymyrss 17
  • XS_gabs model is not correctly imported

    XS_gabs model is not correctly imported

    I am not sure yet what is the cause, but the model takes only 0 or 1 values, no intermediate, independently of the strength. Position and width still work, but fit breaks, naturally, due to lack of sensitivity to strength. cyclabs e.g. works well (but I don't need it now).

    opened by volodymyrss 17
  • Installation with Fermitools

    Installation with Fermitools

    Discussed in https://github.com/threeML/threeML/discussions/576

    Originally posted by PepeJoseHU September 11, 2022 Hello, I plan to install 3ML for Fermi data with CONDA, but looks 3ML have dependence with fermitools. But when I install 'fermitools', it also make an environment with CONDA, so how to fix this problem or if there have way to make connection between different CONDA environments?

    Thanks!

    opened by grburgess 16
  • Response and GBM files

    Response and GBM files

    There seems to be an issue with the reading of GBM DRMs (similar to what I have seen with gammapy)

    
    /usr/local/Cellar/python/HEAD/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/threeML-0.3.2-py2.7.egg/threeML/plugins/ogip.pyc in _read_matrix(self, data, n_channels)
        750             for j in range(n_grp[i]):
        751 
    --> 752                 rsp[i, f_chan[i][j]: f_chan[i][j] + n_chan[i][j]] = matrix[i][m_start:m_start + n_chan[i][j]]
        753 
        754                 m_start += n_chan[i][j]
    
    ValueError: could not broadcast input array from shape (128) into shape (127)
    
    
    
    bug 
    opened by grburgess 16
  • Plotting

    Plotting

    I propose some updates to plotting routines...

    1. Added a function to joint_likelihood.py that will return 1D/2D likelihood profiles/contours for all parameters/combinations of parameters for convenience.

    2. Added an option to plot_point_source_spectra() to also plot extended source spectra (this will just plot whatever was set as the spectral shape, i.e. the 'spatially integrated' spectrum.

    3a. Moved convergence_plots() and plot_chains() from bayesian_analysis.py to analysis_results.py, so that the plots can be produced from analysis results loaded from disk. 3b. Changed convergence_plots() to use the same number of samples for the bootstrapping average as for the sliding window, I hope that was the original intent.

    opened by henrikef 15
  • added upper/lower limits

    added upper/lower limits

    Added three functions based on the equal_tail_intervals to get the X percent upper/lower limits.

    This makes things rather convenient for finding lower limits on cutoffs (or flux normalizations) after doing Bayesian analysis, although I suppose it can also be used in a frequentist analysis once the variates are computed.

    opened by cbrisboi 13
  • "Current minimum stored after fit and current do not correspond!" messages with all PyGMO algorithms

    I am getting "Current minimum stored after fit and current do not correspond!" messages with all PyGMO algorithms for a very simple model and data, which works with MINUIT.

    opened by volodymyrss 12
  • Alternative to root_numpy

    Alternative to root_numpy

    I was looking at updating hawc_hal to use uproot rather than the old ROOT interface for its .root file io and saw that a big part of the root_numpy dependency is carried over from 3ML via the dependency on threeML/io/cern_root_utils. I would generally prefer to stay consistent with file handling between 3ML and hawc_hal. Is there a preference for uproot vs the new ROOT interface? This is already an issue in hawc_hal, see https://github.com/threeML/hawc_hal/issues/42

    The code looks straightforward to update, but before I run off and implement something that is not preferred, I thought asking would be wise first.

    no-issue-activity 
    opened by cbrisboi 11
  • "FitFailed: MIGRAD call failed. This is usually due to unconstrained parameters."

    Hi,

    I'm running into the following fitting error while trying to fit just Fermi-LAT data: "FitFailed: MIGRAD call failed. This is usually due to unconstrained parameters."

    I'm primarily interested in using 3ML to create joint fits of Fermi-LAT data with X-ray (NuSTAR/NICER/XMM/etc) data. I have already downloaded and created intermediate (ltcube, expo maps, etc). To recreate my error: I create a data from the DataList like so:

    eventFile = os.path.join( 'photon_file.fits' ) ft2File = os.path.join('spacecraft_file.fits' ) expomap = os.path.join( 'bexpmap_00.fits' ) ltcube = os.path.join( 'ltcube_00.fits' )

    if is_plugin_available("FermiLATLike"):

    LAT = FermiLATLike( "LAT", eventFile, ft2File, ltcube, 'binned', binned_expo_map=expomap, source_maps=srcmap )
    #I'm performing binned analysis on my source and have created the binned exposure and src maps.
    

    data = DataList(LAT)

    #%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%

    Now I define my spectral model:

    spectrum=Log_parabola() source1=PointSource(test_src,ra=ra,dec=dec,spectral_shape=spectrum)

    model=Model(source1)

    fluxUnit=1/(u.MeVu.cm**2u.s)

    model.test_src.spectrum.main.Log_parabola.alpha.value=-2.7 model.test_src.spectrum.main.Log_parabola.alpha.bounds=(-5.0,-0.5) model.test_src.spectrum.main.Log_parabola.alpha.fix= False

    model.test_src.spectrum.main.Log_parabola.piv.value= 2134*u.MeV model.test_src.spectrum.main.Log_parabola.piv.fix= True

    model.test_src.spectrum.main.Log_parabola.K.value=2.2e-6fluxUnit model.test_src.spectrum.main.Log_parabola.K.bounds=(1e-12fluxUnit,1e1*fluxUnit) model.test_src.spectrum.main.Log_parabola.K.fix= False

    model.test_src.spectrum.main.Log_parabola.beta.value=0.3 model.test_src.spectrum.main.Log_parabola.beta.bounds=(0.0,1.0) model.test_src.spectrum.main.Log_parabola.beta.fix= False

    #%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%

    And start the likelihood analysis: jl = JointLikelihood(model, data) res = jl.fit()

    #%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%

    At this point, I run into the aforementioned error: "FitFailed: MIGRAD call failed. This is usually due to unconstrained parameters."

    When investigating the model, there don't seem to be any constraints on the normalization (but this may be a product of assigning it a unit?):

                                           value min_value max_value  \
    

    test_src.spectrum.main.Log_parabola.K 0.0 0.0 0.0

    Any help would be appreciated.

    opened by aplangrangian 0
  • Graphics bug - Kills kernel

    Graphics bug - Kills kernel

    Describe the bug Importing TriggerSelector from GtBurst AFTER importing threeML causes crashes when viewing lightcurves or other plots. Specifically, after importing threeML then TriggerSelector, if one downloads GBM data, creates a TimeSeriesBuilder, and then goes to view the lightcurve, a Python3.9 window pops up but is unresponsive, and after force quitting it, the kernel also dies.

    NOTE: This is for BUG reports and not questions about generic data analysis.

    To Reproduce

    Run these commands in jupyter notebook in sequential order:

    from threeML import *

    from GtBurst.TriggerSelector import TriggerSelector

    gbm_data = download_GBM_trigger_data("bn090926181",detectors=['n3'])

    ts1 = TimeSeriesBuilder.from_gbm_cspec_or_ctime('n3', cspec_or_ctime_file=gbm_data['n3']["cspec"], rsp_file=gbm_data['n3']["rsp"] )

    fig = ts1.view_lightcurve(-50,100);

    Expected behavior I expect to view a light curve in the notebook, but a python3.9 unresponsive window pops up, and the kernel dies. This is fixed when importing TriggerSelector first, and then importing threeML.

    Screenshots Screen Shot 2022-12-16 at 11 11 28 AM

    Desktop (please complete the following information):

    • OS: iOS
    • Montery version 12.6
    • Apple M1 Pro chip
    opened by nkirschner 1
  • Finishing CastorLike and Ability to Build flux points.

    Finishing CastorLike and Ability to Build flux points.

    Feature Request

    I have been working on cleaning up and speeding up the castorLike plugin which is designed to fit flux points that have been generated by likelihood profiles (ew). However, I never use this feature, so it would be good to have some help.

    Describe the solution you'd like

    1. Clean up plotting
    2. Documentation and examples of usage
    3. We should build a tool that can actually generate these profiles from any spectral fit. Is it possible from just the FitResults alone?

    @henrikef @giacomov @omodei

    You peeps are probably the experts here. @giacomov started it off, so he might have some directions. Perhaps Eric Charles could be helpful as well?

    help wanted plugin issue 
    opened by grburgess 0
  • reuse extended event class files too

    reuse extended event class files too

    I added a line to include the extended LAT files with a different name also to the list of files to be checked if they were previously downloaded already.

    opened by maklinger 1
  • asymmetric error in XY like

    asymmetric error in XY like

    I have spectral points which have asymmetric error and I am hoping to use XY Like to fit those spectral points. However, it seems like XY Like only supports symmetric errors. Is it possible to make XY Like support asymmetric error?

    opened by jasonfan1997 15
  • Value of BIC

    Value of BIC

    Discussed in https://github.com/threeML/threeML/discussions/585

    Originally posted by changxuezhao September 28, 2022 Dear Dr. Burgess, I have two questions for you about 3ML. First of all, the statistical (whether chi2 or pgstat or BIC) obtained using XSPEC will be smaller than 3ML, I think this is the reason for the likelihood, my question is why the likelihood of 3ML is larger than other tools(xspec,rmfit). Second, why can only the line graph of vFv be drawn without data points, and how to add data points to the vFv graph. Thanks and looking forward to your reply。

    no-issue-activity statistics 
    opened by grburgess 1
Releases(v0.1.0)
  • v0.1.0(Apr 19, 2016)

  • v0.0.5(Feb 6, 2015)

    All dependencies are now explicitly stated in the setup.py file, with the exception of ROOT (we might get rid of this dependency and get MINUIT from somewhere else)

    Source code(tar.gz)
    Source code(zip)
  • v0.0.4(Feb 6, 2015)

    The user can install this version by using pip ("pip install threeML").

    The dependencies (numpy, scipy, numexpr, numdifftools...) must be installed separately.

    Source code(tar.gz)
    Source code(zip)
Owner
The Multi-Mission Maximum Likelihood (3ML)
The Multi-Mission Maximum Likelihood (3ML)
Pytorch implementation of XRD spectral identification from COD database

XRDidentifier Pytorch implementation of XRD spectral identification from COD database. Details will be explained in the paper to be submitted to NeurI

Masaki Adachi 4 Jan 07, 2023
Sequence Modeling with Structured State Spaces

Structured State Spaces for Sequence Modeling This repository provides implementations and experiments for the following papers. S4 Efficiently Modeli

HazyResearch 896 Jan 01, 2023
Decoding the Protein-ligand Interactions Using Parallel Graph Neural Networks

Decoding the Protein-ligand Interactions Using Parallel Graph Neural Networks Requirements python 0.10+ rdkit 2020.03.3.0 biopython 1.78 openbabel 2.4

Neeraj Kumar 3 Nov 23, 2022
Train Scene Graph Generation for Visual Genome and GQA in PyTorch >= 1.2 with improved zero and few-shot generalization.

Scene Graph Generation Object Detections Ground truth Scene Graph Generated Scene Graph In this visualization, woman sitting on rock is a zero-shot tr

Boris Knyazev 93 Dec 28, 2022
masscan + nmap + Finger

说明 个人根据使用习惯修改masnmap而来的一个小工具。调用masscan做全端口扫描,再调用nmap做服务识别,最后调用Finger做Web指纹识别。工具使用场景适合风险探测排查、众测等。 使用方法 安装依赖 pip3 install -r requirements.txt -i https:/

Ryan 3 Mar 25, 2022
End-to-end speech secognition toolkit

End-to-end speech secognition toolkit This is an E2E ASR toolkit modified from Espnet1 (version 0.9.9). This is the official implementation of paper:

Jinchuan Tian 147 Dec 28, 2022
Repository for tackling Kaggle Ultrasound Nerve Segmentation challenge using Torchnet.

Ultrasound Nerve Segmentation Challenge using Torchnet This repository acts as a starting point for someone who wants to start with the kaggle ultraso

Qure.ai 46 Jul 18, 2022
Spearmint Bayesian optimization codebase

Spearmint Spearmint is a software package to perform Bayesian optimization. The Software is designed to automatically run experiments (thus the code n

Formerly: Harvard Intelligent Probabilistic Systems Group -- Now at Princeton 1.5k Dec 29, 2022
Wordplay, an artificial Intelligence based crossword puzzle solver.

Wordplay, AI based crossword puzzle solver A crossword is a word puzzle that usually takes the form of a square or a rectangular grid of white- and bl

Vaibhaw 4 Nov 16, 2022
TART - A PyTorch implementation for Transition Matrix Representation of Trees with Transposed Convolutions

TART This project is a PyTorch implementation for Transition Matrix Representati

Lee Sael 2 Jan 19, 2022
Unofficial PyTorch Implementation of "Augmenting Convolutional networks with attention-based aggregation"

Pytorch Implementation of Augmenting Convolutional networks with attention-based aggregation This is the unofficial PyTorch Implementation of "Augment

DK 20 Sep 09, 2022
This script runs neural style transfer against the provided content image.

Neural Style Transfer Content Style Output Description: This script runs neural style transfer against the provided content image. The content image m

Martynas Subonis 0 Nov 25, 2021
YOLOv5🚀 reproduction by Guo Quanhao using PaddlePaddle

YOLOv5-Paddle YOLOv5 🚀 reproduction by Guo Quanhao using PaddlePaddle 支持AutoBatch 支持AutoAnchor 支持GPU Memory 快速开始 使用AIStudio高性能环境快速构建YOLOv5训练(PaddlePa

QuanHao Guo 20 Nov 14, 2022
UmlsBERT: Clinical Domain Knowledge Augmentation of Contextual Embeddings Using the Unified Medical Language System Metathesaurus

UmlsBERT: Clinical Domain Knowledge Augmentation of Contextual Embeddings Using the Unified Medical Language System Metathesaurus General info This is

71 Oct 25, 2022
clustimage is a python package for unsupervised clustering of images.

clustimage The aim of clustimage is to detect natural groups or clusters of images. Image recognition is a computer vision task for identifying and ve

Erdogan Taskesen 52 Jan 02, 2023
The Unsupervised Reinforcement Learning Benchmark (URLB)

The Unsupervised Reinforcement Learning Benchmark (URLB) URLB provides a set of leading algorithms for unsupervised reinforcement learning where agent

259 Dec 26, 2022
Bayesian algorithm execution (BAX)

Bayesian Algorithm Execution (BAX) Code for the paper: Bayesian Algorithm Execution: Estimating Computable Properties of Black-box Functions Using Mut

Willie Neiswanger 38 Dec 08, 2022
[NeurIPS 2021] SSUL: Semantic Segmentation with Unknown Label for Exemplar-based Class-Incremental Learning

SSUL - Official Pytorch Implementation (NeurIPS 2021) SSUL: Semantic Segmentation with Unknown Label for Exemplar-based Class-Incremental Learning Sun

Clova AI Research 44 Dec 27, 2022
Anonymize BLM Protest Images

Anonymize BLM Protest Images This repository automates @BLMPrivacyBot, a Twitter bot that shows the anonymized images to help keep protesters safe. Us

Stanford Machine Learning Group 40 Oct 13, 2022
Source code of D-HAN: Dynamic News Recommendation with Hierarchical Attention Network

D-HAN The source code of D-HAN This is the source code of D-HAN: Dynamic News Recommendation with Hierarchical Attention Network. However, only the co

30 Sep 22, 2022