The Multi-Mission Maximum Likelihood framework (3ML)

Overview

CI Conda Build and Publish Test Against XSPEC codecov Documentation Status License

GitHub pull requests GitHub issues

PyPi

PyPI version fury.io PyPI - Downloads PyPI - Python Version

Conda

Conda Conda

drawing

The Multi-Mission Maximum Likelihood framework (3ML)

A framework for multi-wavelength/multi-messenger analysis for astronomy/astrophysics.


Astrophysical sources are observed by different instruments at different wavelengths with an unprecedented quality. Putting all these data together to form a coherent view, however, is a very difficult task. Indeed, each instrument and data type has its own ad-hoc software and handling procedure, which present steep learning curves and do not talk to each other.

The Multi-Mission Maximum Likelihood framework (3ML) provides a common high-level interface and model definition, which allows for an easy, coherent and intuitive modeling of sources using all the available data, no matter their origin. At the same time, thanks to its architecture based on plug-ins, 3ML uses under the hood the official software of each instrument, the only one certified and maintained by the collaboration which built the instrument itself. This guarantees that 3ML is always using the best possible methodology to deal with the data of each instrument.

drawing

Though Maximum Likelihood is in the name for historical reasons, 3ML is an interface to several Bayesian inference algorithms such as MCMC and nested sampling as well as likelihood optimization algorithms. Each approach to analysis can be seamlessly switched between allowing users to try different approaches quickly and without having to rewrite their model or data interfaces.

Like your XPSEC models? You can use them in 3ML as well as our growing selection of 1-,2- and 3-D models from our fast and customizable modeling language astromodels.

Installation

Installing with pip or conda is easy

pip install astromodels threeml
conda  install astromodels threeml -c threeml conda-forge 

Please refer to the Installation instructions for more details and trouble-shooting.

Press

Who is using 3ML?

Here is a highlight list of teams and their publications using 3ML.

A full list of publications using 3ML is here.

Citing

If you find this package useful in you analysis, or the code in your own custom data tools, please cite:

Vianello et al. (2015)

Acknowledgements

3ML makes use of the Spanish Virtual Observatory's Filter Profile service (http://svo2.cab.inta-csic.es/svo/theory/fps3/index.php?mode=browse&gname=NIRT).

If you use these profiles in your research, please consider citing them by using the following suggested sentence in your paper:

"This research has made use of the SVO Filter Profile Service (http://svo2.cab.inta-csic.es/theory/fps/) supported from the Spanish MINECO through grant AyA2014-55216"

and citing the following publications:

The SVO Filter Profile Service. Rodrigo, C., Solano, E., Bayo, A. http://ivoa.net/documents/Notes/SVOFPS/index.html The Filter Profile Service Access Protocol. Rodrigo, C., Solano, E. http://ivoa.net/documents/Notes/SVOFPSDAL/index.html

ThreeML is supported by National Science Foundation (NSF)

Comments
  • Use from astromodels Xspec import * error

    Use from astromodels Xspec import * error

    Hello! When I want to use xspec library to combine GBM and XRT to analyze energy spectrum, I try to use from astromodels Xspec import *has the following error messages. How did this happen?


    ImportError Traceback (most recent call last) /tmp/ipykernel_12482/3995023305.py in 5 6 # we will need XPSEC models for extinction ----> 7 from astromodels.xspec import * 8 from astromodels.xspec.xspec_settings import *

    ~/miniconda3/envs/threeML/lib/python3.7/site-packages/astromodels/xspec/init.py in ----> 1 from .factory import * 2 #from .xspec_settings import x

    ~/miniconda3/envs/threeML/lib/python3.7/site-packages/astromodels/xspec/factory.py in 16 pass 17 ---> 18 from astromodels.xspec import _xspec 19 20 # When running in a Anaconda environment, the package xspec-modelsonly

    ImportError: cannot import name '_xspec' from 'astromodels.xspec' (/home/hank/miniconda3/envs/threeML/lib/python3.7/site-packages/astromodels/xspec/init.py)

    opened by ghost 64
  • How do I install a stable threeml module?

    How do I install a stable threeml module?

    Hello,

    The threeml module is currently latest, causing many modules to fail to install, including the Xspexcmodule. So I want to install a more stable threeml module, so how do I do that?

    opened by ghost 18
  • Background Modeling

    Background Modeling

    I've been looking at the background modeling and it may be better to approach it by adding a node onto PointSource:

    ps = PointSource(...,background_shape=Powerlaw() )
    

    Otherwise, we are just adding a function inside of the plugin and have to dig into private members to access the free parameters of that function.

    Is this too involved of an approach @giacomov ?

    opened by grburgess 18
  • properties of the spectra and choosing sources for SpectrumLike

    properties of the spectra and choosing sources for SpectrumLike

    would it make sense to assign some properties in the SpectrumLike plugin. it is quite useful when one wants to associate only some sources to a given spectrum, instead of summing all, as it is done now.

    opened by volodymyrss 17
  • XS_gabs model is not correctly imported

    XS_gabs model is not correctly imported

    I am not sure yet what is the cause, but the model takes only 0 or 1 values, no intermediate, independently of the strength. Position and width still work, but fit breaks, naturally, due to lack of sensitivity to strength. cyclabs e.g. works well (but I don't need it now).

    opened by volodymyrss 17
  • Installation with Fermitools

    Installation with Fermitools

    Discussed in https://github.com/threeML/threeML/discussions/576

    Originally posted by PepeJoseHU September 11, 2022 Hello, I plan to install 3ML for Fermi data with CONDA, but looks 3ML have dependence with fermitools. But when I install 'fermitools', it also make an environment with CONDA, so how to fix this problem or if there have way to make connection between different CONDA environments?

    Thanks!

    opened by grburgess 16
  • Response and GBM files

    Response and GBM files

    There seems to be an issue with the reading of GBM DRMs (similar to what I have seen with gammapy)

    
    /usr/local/Cellar/python/HEAD/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/threeML-0.3.2-py2.7.egg/threeML/plugins/ogip.pyc in _read_matrix(self, data, n_channels)
        750             for j in range(n_grp[i]):
        751 
    --> 752                 rsp[i, f_chan[i][j]: f_chan[i][j] + n_chan[i][j]] = matrix[i][m_start:m_start + n_chan[i][j]]
        753 
        754                 m_start += n_chan[i][j]
    
    ValueError: could not broadcast input array from shape (128) into shape (127)
    
    
    
    bug 
    opened by grburgess 16
  • Plotting

    Plotting

    I propose some updates to plotting routines...

    1. Added a function to joint_likelihood.py that will return 1D/2D likelihood profiles/contours for all parameters/combinations of parameters for convenience.

    2. Added an option to plot_point_source_spectra() to also plot extended source spectra (this will just plot whatever was set as the spectral shape, i.e. the 'spatially integrated' spectrum.

    3a. Moved convergence_plots() and plot_chains() from bayesian_analysis.py to analysis_results.py, so that the plots can be produced from analysis results loaded from disk. 3b. Changed convergence_plots() to use the same number of samples for the bootstrapping average as for the sliding window, I hope that was the original intent.

    opened by henrikef 15
  • added upper/lower limits

    added upper/lower limits

    Added three functions based on the equal_tail_intervals to get the X percent upper/lower limits.

    This makes things rather convenient for finding lower limits on cutoffs (or flux normalizations) after doing Bayesian analysis, although I suppose it can also be used in a frequentist analysis once the variates are computed.

    opened by cbrisboi 13
  • "Current minimum stored after fit and current do not correspond!" messages with all PyGMO algorithms

    I am getting "Current minimum stored after fit and current do not correspond!" messages with all PyGMO algorithms for a very simple model and data, which works with MINUIT.

    opened by volodymyrss 12
  • Alternative to root_numpy

    Alternative to root_numpy

    I was looking at updating hawc_hal to use uproot rather than the old ROOT interface for its .root file io and saw that a big part of the root_numpy dependency is carried over from 3ML via the dependency on threeML/io/cern_root_utils. I would generally prefer to stay consistent with file handling between 3ML and hawc_hal. Is there a preference for uproot vs the new ROOT interface? This is already an issue in hawc_hal, see https://github.com/threeML/hawc_hal/issues/42

    The code looks straightforward to update, but before I run off and implement something that is not preferred, I thought asking would be wise first.

    no-issue-activity 
    opened by cbrisboi 11
  • "FitFailed: MIGRAD call failed. This is usually due to unconstrained parameters."

    Hi,

    I'm running into the following fitting error while trying to fit just Fermi-LAT data: "FitFailed: MIGRAD call failed. This is usually due to unconstrained parameters."

    I'm primarily interested in using 3ML to create joint fits of Fermi-LAT data with X-ray (NuSTAR/NICER/XMM/etc) data. I have already downloaded and created intermediate (ltcube, expo maps, etc). To recreate my error: I create a data from the DataList like so:

    eventFile = os.path.join( 'photon_file.fits' ) ft2File = os.path.join('spacecraft_file.fits' ) expomap = os.path.join( 'bexpmap_00.fits' ) ltcube = os.path.join( 'ltcube_00.fits' )

    if is_plugin_available("FermiLATLike"):

    LAT = FermiLATLike( "LAT", eventFile, ft2File, ltcube, 'binned', binned_expo_map=expomap, source_maps=srcmap )
    #I'm performing binned analysis on my source and have created the binned exposure and src maps.
    

    data = DataList(LAT)

    #%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%

    Now I define my spectral model:

    spectrum=Log_parabola() source1=PointSource(test_src,ra=ra,dec=dec,spectral_shape=spectrum)

    model=Model(source1)

    fluxUnit=1/(u.MeVu.cm**2u.s)

    model.test_src.spectrum.main.Log_parabola.alpha.value=-2.7 model.test_src.spectrum.main.Log_parabola.alpha.bounds=(-5.0,-0.5) model.test_src.spectrum.main.Log_parabola.alpha.fix= False

    model.test_src.spectrum.main.Log_parabola.piv.value= 2134*u.MeV model.test_src.spectrum.main.Log_parabola.piv.fix= True

    model.test_src.spectrum.main.Log_parabola.K.value=2.2e-6fluxUnit model.test_src.spectrum.main.Log_parabola.K.bounds=(1e-12fluxUnit,1e1*fluxUnit) model.test_src.spectrum.main.Log_parabola.K.fix= False

    model.test_src.spectrum.main.Log_parabola.beta.value=0.3 model.test_src.spectrum.main.Log_parabola.beta.bounds=(0.0,1.0) model.test_src.spectrum.main.Log_parabola.beta.fix= False

    #%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%

    And start the likelihood analysis: jl = JointLikelihood(model, data) res = jl.fit()

    #%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%

    At this point, I run into the aforementioned error: "FitFailed: MIGRAD call failed. This is usually due to unconstrained parameters."

    When investigating the model, there don't seem to be any constraints on the normalization (but this may be a product of assigning it a unit?):

                                           value min_value max_value  \
    

    test_src.spectrum.main.Log_parabola.K 0.0 0.0 0.0

    Any help would be appreciated.

    opened by aplangrangian 0
  • Graphics bug - Kills kernel

    Graphics bug - Kills kernel

    Describe the bug Importing TriggerSelector from GtBurst AFTER importing threeML causes crashes when viewing lightcurves or other plots. Specifically, after importing threeML then TriggerSelector, if one downloads GBM data, creates a TimeSeriesBuilder, and then goes to view the lightcurve, a Python3.9 window pops up but is unresponsive, and after force quitting it, the kernel also dies.

    NOTE: This is for BUG reports and not questions about generic data analysis.

    To Reproduce

    Run these commands in jupyter notebook in sequential order:

    from threeML import *

    from GtBurst.TriggerSelector import TriggerSelector

    gbm_data = download_GBM_trigger_data("bn090926181",detectors=['n3'])

    ts1 = TimeSeriesBuilder.from_gbm_cspec_or_ctime('n3', cspec_or_ctime_file=gbm_data['n3']["cspec"], rsp_file=gbm_data['n3']["rsp"] )

    fig = ts1.view_lightcurve(-50,100);

    Expected behavior I expect to view a light curve in the notebook, but a python3.9 unresponsive window pops up, and the kernel dies. This is fixed when importing TriggerSelector first, and then importing threeML.

    Screenshots Screen Shot 2022-12-16 at 11 11 28 AM

    Desktop (please complete the following information):

    • OS: iOS
    • Montery version 12.6
    • Apple M1 Pro chip
    opened by nkirschner 1
  • Finishing CastorLike and Ability to Build flux points.

    Finishing CastorLike and Ability to Build flux points.

    Feature Request

    I have been working on cleaning up and speeding up the castorLike plugin which is designed to fit flux points that have been generated by likelihood profiles (ew). However, I never use this feature, so it would be good to have some help.

    Describe the solution you'd like

    1. Clean up plotting
    2. Documentation and examples of usage
    3. We should build a tool that can actually generate these profiles from any spectral fit. Is it possible from just the FitResults alone?

    @henrikef @giacomov @omodei

    You peeps are probably the experts here. @giacomov started it off, so he might have some directions. Perhaps Eric Charles could be helpful as well?

    help wanted plugin issue 
    opened by grburgess 0
  • reuse extended event class files too

    reuse extended event class files too

    I added a line to include the extended LAT files with a different name also to the list of files to be checked if they were previously downloaded already.

    opened by maklinger 1
  • asymmetric error in XY like

    asymmetric error in XY like

    I have spectral points which have asymmetric error and I am hoping to use XY Like to fit those spectral points. However, it seems like XY Like only supports symmetric errors. Is it possible to make XY Like support asymmetric error?

    opened by jasonfan1997 15
  • Value of BIC

    Value of BIC

    Discussed in https://github.com/threeML/threeML/discussions/585

    Originally posted by changxuezhao September 28, 2022 Dear Dr. Burgess, I have two questions for you about 3ML. First of all, the statistical (whether chi2 or pgstat or BIC) obtained using XSPEC will be smaller than 3ML, I think this is the reason for the likelihood, my question is why the likelihood of 3ML is larger than other tools(xspec,rmfit). Second, why can only the line graph of vFv be drawn without data points, and how to add data points to the vFv graph. Thanks and looking forward to your reply。

    no-issue-activity statistics 
    opened by grburgess 1
Releases(v0.1.0)
  • v0.1.0(Apr 19, 2016)

  • v0.0.5(Feb 6, 2015)

    All dependencies are now explicitly stated in the setup.py file, with the exception of ROOT (we might get rid of this dependency and get MINUIT from somewhere else)

    Source code(tar.gz)
    Source code(zip)
  • v0.0.4(Feb 6, 2015)

    The user can install this version by using pip ("pip install threeML").

    The dependencies (numpy, scipy, numexpr, numdifftools...) must be installed separately.

    Source code(tar.gz)
    Source code(zip)
Owner
The Multi-Mission Maximum Likelihood (3ML)
The Multi-Mission Maximum Likelihood (3ML)
To prepare an image processing model to classify the type of disaster based on the image dataset

Disaster Classificiation using CNNs bunnysaini/Disaster-Classificiation Goal To prepare an image processing model to classify the type of disaster bas

Bunny Saini 1 Jan 24, 2022
Official code for MPG2: Multi-attribute Pizza Generator: Cross-domain Attribute Control with Conditional StyleGAN

This is the official code for Multi-attribute Pizza Generator (MPG2): Cross-domain Attribute Control with Conditional StyleGAN. Paper Demo Setup Envir

Fangda Han 5 Sep 01, 2022
ADOP: Approximate Differentiable One-Pixel Point Rendering

ADOP: Approximate Differentiable One-Pixel Point Rendering Abstract: We present a novel point-based, differentiable neural rendering pipeline for scen

Darius Rückert 1.9k Jan 06, 2023
Single Image Random Dot Stereogram for Tensorflow

TensorFlow-SIRDS Single Image Random Dot Stereogram for Tensorflow SIRDS is a means to present 3D data in a 2D image. It allows for scientific data di

Greg Peatfield 5 Aug 10, 2022
Face Recognition plus identification simply and fast | Python

PyFaceDetection Face Recognition plus identification simply and fast Ubuntu Setup sudo pip3 install numpy sudo pip3 install cmake sudo pip3 install dl

Peyman Majidi Moein 16 Sep 22, 2022
Tensorflow implementation of ID-Unet: Iterative Soft and Hard Deformation for View Synthesis.

ID-Unet: Iterative-view-synthesis(CVPR2021 Oral) Tensorflow implementation of ID-Unet: Iterative Soft and Hard Deformation for View Synthesis. Overvie

17 Aug 23, 2022
This repository is for our EMNLP 2021 paper "Automated Generation of Accurate & Fluent Medical X-ray Reports"

Introduction: X-Ray Report Generation This repository is for our EMNLP 2021 paper "Automated Generation of Accurate & Fluent Medical X-ray Reports". O

no name 36 Dec 16, 2022
A diff tool for language models

LMdiff Qualitative comparison of large language models. Demo & Paper: http://lmdiff.net LMdiff is a MIT-IBM Watson AI Lab collaboration between: Hendr

Hendrik Strobelt 27 Dec 29, 2022
Unofficial PyTorch implementation of Masked Autoencoders Are Scalable Vision Learners

Unofficial PyTorch implementation of Masked Autoencoders Are Scalable Vision Learners This repository is built upon BEiT, thanks very much! Now, we on

Zhiliang Peng 2.3k Jan 04, 2023
Pytorch Implementation of LNSNet for Superpixel Segmentation

LNSNet Overview Official implementation of Learning the Superpixel in a Non-iterative and Lifelong Manner (CVPR'21) Learning Strategy The proposed LNS

42 Oct 11, 2022
Goal of the project : Detecting Temporal Boundaries in Sign Language videos

MVA RecVis course final project : Goal of the project : Detecting Temporal Boundaries in Sign Language videos. Sign language automatic indexing is an

Loubna Ben Allal 6 Dec 21, 2022
Optimal space decomposition based-product quantization for approximate nearest neighbor search

Optimal space decomposition based-product quantization for approximate nearest neighbor search Abstract Product quantization(PQ) is an effective neare

Mylove 1 Nov 19, 2021
An implementation of Geoffrey Hinton's paper "How to represent part-whole hierarchies in a neural network" in Pytorch.

GLOM An implementation of Geoffrey Hinton's paper "How to represent part-whole hierarchies in a neural network" for MNIST Dataset. To understand this

50 Oct 19, 2022
Pytorch implementation of the paper "Optimization as a Model for Few-Shot Learning"

Optimization as a Model for Few-Shot Learning This repo provides a Pytorch implementation for the Optimization as a Model for Few-Shot Learning paper.

Albert Berenguel Centeno 238 Jan 04, 2023
Pytorch implementations of popular off-policy multi-agent reinforcement learning algorithms, including QMix, VDN, MADDPG, and MATD3.

Off-Policy Multi-Agent Reinforcement Learning (MARL) Algorithms This repository contains implementations of various off-policy multi-agent reinforceme

183 Dec 28, 2022
Keras community contributions

keras-contrib : Keras community contributions Keras-contrib is deprecated. Use TensorFlow Addons. The future of Keras-contrib: We're migrating to tens

Keras 1.6k Dec 21, 2022
Convert onnx models to pytorch.

onnx2torch onnx2torch is an ONNX to PyTorch converter. Our converter: Is easy to use – Convert the ONNX model with the function call convert; Is easy

ENOT 264 Dec 30, 2022
A PyTorch implementation of the paper "Semantic Image Synthesis via Adversarial Learning" in ICCV 2017

Semantic Image Synthesis via Adversarial Learning This is a PyTorch implementation of the paper Semantic Image Synthesis via Adversarial Learning. Req

Seonghyeon Nam 146 Nov 25, 2022
Faune proche - Retrieval of Faune-France data near a google maps location

faune_proche Récupération des données de Faune-France près d'un lieu google maps

4 Feb 15, 2022
A curated list of awesome papers for Semantic Retrieval (TOIS Accepted: Semantic Models for the First-stage Retrieval: A Comprehensive Review).

A curated list of awesome papers for Semantic Retrieval (TOIS Accepted: Semantic Models for the First-stage Retrieval: A Comprehensive Review).

Yinqiong Cai 189 Dec 28, 2022