Multifunctional Analysis of Regions through Input-Output

Related tags

MiscellaneousMARIO
Overview
https://img.shields.io/gitter/room/SESAM-Polimi/MARIO Documentation Status https://img.shields.io/pypi/v/mariopy https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png

MARIO

Multifunctional Analysis of Regions through Input-Output. (Documents)

What is it

MARIO is a python package for handling input-output tables and models inspired by pymrio. MARIO aims to provide a simple & intuitive API for common IO tasks without needing in-depth programming knowledge. MARIO supporst automatic parsing of different structured tables such EXIOBASE, EORA, EUROSTAT in different formats namely:

  • Single region
  • Multi region
  • Hybrid tables
  • Monetary tables
  • Input-Output tables
  • Supply-Use tables

When databases are not structured, MARIO supports parsing data from xlsx, csv, txt files or pandas.DataFrames.

More than parsing data, MARIO includes some basic functionalities:

  • Aggregation of databases
  • SUT to IOT transformation
  • Modifying database in terms of adding:
    • New sectors, activities or commodities to the database
    • Adding new extensions to the satellite account
  • Scneario and shock analysis
  • Backward and forward linkages analysis
  • Extracting single region database from multi region databases
  • Balance test
  • Productivity test
  • Exporting the databases into different formats for scenarios analyzed
  • Interactive visualization routines

Requirements

MARIO has been tested on macOS and Windows.

To run MARIO, a couple of things are needed:

  1. Being in love with Input-Output :-)
  2. The Python programming language, version 3.7 or higher
  3. A number of Python adds-on packages
  4. For some functionalities a solver may needed (optional)
  5. MARIO software itself

Recommended installation method

The easiest way to make MARIO software working is to use the free conda package manager which can install the current and future MARIO depencies in an easy and user friendly way.

To get conda, download and install "Anaconda Distribution" . Between differnet options for running python codes, we strongly suggest, Spyder, which is a free and open source scientific environment written in Python, for Python, and designed by and for scientists, engineers and data analysts.

You can install mario using pip or from source code. It is suggested to create a new environment by running the following command in the anaconda prompt

conda create -n mario python=3.8

If you create a new environment for mario, to use it, you need to activate the mario environment each time by writing the following line in Anaconda Prompt

conda activate mario

Now you can use pip to install mario on your environment as follow:

pip install mariopy

You can also install from the source code!

Quickstart

A simple test for Input-Output Table (IOT) and Supply-Use Table (SUT) is included in mario.

To use the IOT test, call

import mario
test_iot = mario.load_test('IOT')

and to use the SUT test, call

test_sut = mario.load_test('SUT')

To see the configurations of the data, you can print them:

print(test_iot)
print(test_sut)

To see specific sets of the tables like regions or value added, get_index function can be used:

print(test_iot.get_index('Region'))
print(test_sut.get_index('Factor of production'))

To visualize some data, various plot functions can be used:

test_iot.plot_matrix(....)

Specific modifications on the database can be done, such as SUT to IOT transformation:

reformed_iot = test.sut_to_iot(method='B')

The changes can be tracked by metadata. The history can be checked by calling:

reformed_iot.meta_history

The new database can be saved into excel,txt or csv file:

reformed_iot.to_excel(path='a folder//database.xlsx')

Python module requirements

Some of the key packages the mario relies on are:

Note

  • This project is under active development.
  • More examples will be uploaded through time to the gellery.
  • More parsers will be added to the next version.
  • The next version will cover some optimization models within the IO framework

License

https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png

This work is licensed under a Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0)

Comments
  • Misleading error message for `parse_eurostat`

    Misleading error message for `parse_eurostat`

    Following the documentation for the Eurostat-parsing functionality, I ran into below error ("non-corresponding years"). This is unexpected since I was strictly following the documentation, i.e. did everything as described there. Any ideas why that happens and how to get my SUTs?

    `--------------------------------------------------------------------------- WrongInput Traceback (most recent call last) c:\Users\maximiko\IO-testbed\Eurostat_MARIO.py in ----> 18 italy_sut = parse_eurostat( 19 supply_path = supply_path, 20 use_path = use_path, 21 region = "Italy", 22 year = 2011

    ~\Anaconda3\envs\pymrio\lib\site-packages\mario\tools\parsersclass.py in parse_eurostat(supply_path, use_path, region, year, consumption_categories, factors_of_production, imports, model, name, calc_all, **kwargs) 432 table = "SUT" 433 if table == "SUT": --> 434 matrices, indeces, units = eurostat_sut( 435 supply_path, 436 use_path,

    ~\Anaconda3\envs\pymrio\lib\site-packages\mario\tools\tableparser.py in eurostat_sut(supply_path, use_path, region, selected_year, consumption_categories, factors_of_production, imports) 992 years = supply_years 993 else: --> 994 raise WrongInput("No correspondance between years of use and supply tables") 995 996 "Warning on supply in IxP format, Use in PxI format"

    WrongInput: No correspondance between years of use and supply tables`

    enhancement 
    opened by maximikos 4
  • New parse hybrid Exiobase from Zenodo

    New parse hybrid Exiobase from Zenodo

    Now the Hybrid version of EXIOBASE is available on Zenodo at https://zenodo.org/record/7244919#.Y6Lr0NXMIQ8

    A new parse function is needed in order to take advantage of a lighter format.

    enhancement 
    opened by nigolred 1
  • calc all keyerror management

    calc all keyerror management

    In the calc_all function, we should check at the beginning of the function whether the scenario exists or not so the keyerror won't be about the missing scenario.

    opened by LorenzoRinaldi 1
  • search function minor improvement

    search function minor improvement

    The search function should not be case sensitive. For instance, if the user run this command:

    db.search("Sector","electricity")

    he will not be able to track for sectors named "Electricity..." with capital "E". It would be super-useful to improve this making the function not case-sensitive

    opened by LorenzoRinaldi 1
  • Pymrio integration

    Pymrio integration

    pull request for merging the functions and methods added for mario vs pymrio integration.

    Changes:

    • Adding a new method for converting mario.Database to pymrio.IOSystem in Database class
    • Adding the parser function to parsers that reshapes the pymrio.IOSystem and pymrio.Extensions to mario.Database
    • miscellaneous changes in tools and constants to map the differences in two libraries
    opened by mohammadamint 1
  • There should be a way to store the metadata that generated a new database

    There should be a way to store the metadata that generated a new database

    Every time a new database is printed-out there should be a way to access information on the:

    • year
    • table version
    • added extensions
    • ...

    Printing out the metadata (as a txt file in case .to_txt or .to_csv is used; as an additional sheet in case .to_excel is used) can do the job

    opened by nigolred 1
  • Add a Gitter chat badge to README.rst

    Add a Gitter chat badge to README.rst

    SESAM-Polimi/MARIO now has a Chat Room on Gitter

    @LorenzoRinaldi has just created a chat room. You can visit it here: https://gitter.im/SESAM-Polimi/MARIO.

    This pull-request adds this badge to your README.rst:

    Gitter

    If my aim is a little off, please let me know.

    Happy chatting.

    PS: Click here if you would prefer not to receive automatic pull-requests from Gitter in future.

    opened by gitter-badger 1
  • Add pymrio package to requirements

    Add pymrio package to requirements

    TLDR I tried to follow the Mario set up instruction and I believe the pymrio package is missing from requirements.txt. I am using this as an opportunity to try my first pull request on the MARIO repo just to make sure everything works properly.

    REPRO

    i tried to install MARIO and then run a simple script using it. Here is the script I tried: https://gist.github.com/AAmedeo/fdeb1f74413d109d3f2727fa1116e687

    This is what happened before adding pymrio to requirements.txt

    [email protected] MARIO % rm -r venv
    [email protected] MARIO % python3 -m venv venv
    [email protected] MARIO % source ./venv/bin/activate
    (venv) [email protected] MARIO % pip3 install -r requirements.txt
    (venv) [email protected] MARIO % python3 mario_hello_world.py
    Traceback (most recent call last):
      File "mario_hello_world.py", line 1, in <module>
        import mario
      File "/Users/afa/Desktop/MARIO/mario/__init__.py", line 48, in <module>
        from mario.core.AttrData import Database
      File "/Users/afa/Desktop/MARIO/mario/core/AttrData.py", line 91, in <module>
        from mario.core.CoreIO import CoreModel
      File "/Users/afa/Desktop/MARIO/mario/core/CoreIO.py", line 15, in <module>
        from mario.tools.tableparser import dataframe_parser
      File "/Users/afa/Desktop/MARIO/mario/tools/tableparser.py", line 48, in <module>
        import pymrio
    ModuleNotFoundError: No module named 'pymrio'
    

    This is what happened after adding pymrio to requirements.txt

    [email protected] MARIO % rm -r venv
    [email protected] MARIO % python3 -m venv venv
    [email protected] MARIO % source ./venv/bin/activate
    (venv) [email protected] MARIO % pip3 install -r requirements.txt
    (venv) [email protected] MARIO % python3 mario_hello_world.py
    cvxpy module is not installed in your system. This will raise problems in some of the abilities of MARIO
    name = IOT test
    table = IOT
    scenarios = ['baseline']
    Factor of production = 3
    Satellite account = 4
    Consumption category = 1
    Region = 2
    Sector = 6
    
    name = SUT test
    table = SUT
    scenarios = ['baseline']
    Activity = 6
    Commodity = 6
    Factor of production = 3
    Satellite account = 1
    Consumption category = 1
    Region = 2
    
    ['RoW', 'Italy']
    ['Taxes', 'Wages', 'Capital']
    
    opened by AAmedeo 0
  • Add sectors conflict with pandas 1.4.4. version

    Add sectors conflict with pandas 1.4.4. version

    Using add_sectors function with pandas version 1.4.4 lead to the following error:

    ValueError: Length of new names must be 1, got 3

    The function works with pandas 1.3.3

    dependencies 
    opened by LorenzoRinaldi 1
  • Adding parsing function for Exiobase 3.3.18 (hybrid) version

    Adding parsing function for Exiobase 3.3.18 (hybrid) version

    I am requesting to merge these changes to main branch. The changes implement a new parser function for exiobase database, in particular referring to version 3.3.18, downloadable at this link

    In particular, the new feature have been implemented by updating the already present function parse_exiobase_sut, by adding the unit, version and extensions attributes.

    The unit attribute can be selected among the _acceptable_units list, saved in the parserid.py file, and is by default set to "monetary". In the default case, the function is run exactly in the same way as the one in the main branch. The version attribute is by default set to None and must be provided in case unit="hybrid" The extensions attribute is by default set to None and can be provided as a list including any of the keys of the _acceptable_extensions dictionay saved in the parserid.py file.

    Regarding the case when unit="hybrid", then the new hybrid_sut_parser function (in tableparser.py) will be invoked (see commit 799443fd5b3e0944894238b0d4c5861fe8e18640).

    Initially, this function was developed to parse an old version of the database, downloadable from the official exiobase website, which was saved as a zip file. This version had supply, use and final demand matrices saved into separate .xlsb files therefore the parser was significantly slower than other ones for other exiobase versions. The function was then modified in order to parse the database as the one available on zenodo (link reported previously), where the same matrices are available by default in .csv formats (much faster to be read).

    Another improvement brought by this merge would be the fact that the aggregation excel file template obtained by the get_aggregation_excel function, automatically prints a new sheet showing the units of each entry of each set. This would ease the process of aggregation in case of hybrid databases.

    enhancement 
    opened by LorenzoRinaldi 0
  • Obsolete xlsxwriter dependency

    Obsolete xlsxwriter dependency

    I see the package depency states "xlsxwriter <= 1.3.7", whereas the current version is 3.0.3... Is this really required? When installing the package, it downgraded my package version, and broke functionnalities with other Python scripts I'm running. Thanks in advance...

    enhancement 
    opened by adrienjacob 1
  • Get_data function

    Get_data function

    The function "get_data" is not working if the user provide an integer as "scenarios" parameter. Not even if it is provided within a list. Moreover, the documentation for this function seems missing.

    bug documentation 
    opened by LorenzoRinaldi 0
  • Activity units

    Activity units

    Check if actvity units is making sense, in general: do they need to be provided with units? Especially in hybrid database the answer is probably no, while for economic databases it is the opposite.

    Also, hybrid database need for further checks when dealing with aggregation of activities due to the fact activities may produce commodities with different units

    opened by LorenzoRinaldi 0
Releases(v0.1.0)
A collection of Python library code for building Python applications.

Abseil Python Common Libraries This repository is a collection of Python library code for building Python applications. The code is collected from Goo

Abseil 2k Jan 07, 2023
Convert three types of color in your clipboard and paste it to the color property (gamma correct)

ColorPaster [Blender Addon] Convert three types of color in your clipboard and paste it to the color property (gamma correct) How to Use Hover your mo

13 Oct 31, 2022
Set of scripts that schedules employees for shifts throughout the week based on availability, shift times, and shift necessities

Automatic-Scheduler Set of scripts that schedules employees for shifts throughout the week based on availability, shift times, and shift necessities *

Matthew 1 May 01, 2022
【教程】莉沫酱教你学继承!?

【教程】莉沫酱教你学继承! 众所周知,类的继承就是说当一个类死亡的时候,它的子类会获得它拥有的资源。 根据类的继承法不同,各个子类能获得的资源也不同。 继承法的类型 在解释继承法之前,我们先定义三个类,一个父类A,和它的子类B、C。 它们都拥有x、y、z三个属性。

黄巍 17 Dec 05, 2022
The presented desktop application was made to solve 1d schrodinger eqation

schrodinger_equation_1d_solver The presented desktop application was made to solve 1d schrodinger eqation. It implements Numerov's algorithm (step by

Artem Kashapov 2 Dec 29, 2021
Age of Empires II recorded game parsing and summarization in Python 3.

mgz Age of Empires II recorded game parsing and summarization in Python 3. Supported Versions Age of Kings (.mgl) The Conquerors (.mgx) Userpatch 1.4

148 Dec 11, 2022
Funchacks - Fun module which is a small set of utilities

funchacks 👋 Introduction Funchacks is a fun module that provides a small packag

DenyS 6 Aug 04, 2022
A tool for removing PUPs using signatures

Unwanted program removal tool A tool for removing PUPs using signatures What is the unwanted program removal tool? The unwanted program removal tool i

4 Sep 20, 2022
ARK sõidueksami Matrixi bot

ARK Sõidueksami bot Küsib ARK-i lehelt uusimad eksami ajad ja saadab sõnumi Matrixi kanali Dev setup Linux python3 -m venv venv source venv/bin/activa

Arti Zirk 3 Jun 15, 2021
CoreSE - basic of social Engineering tool

Core Social Engineering basic of social Engineering tool. just for fun :) About First of all, I must say that I wrote such a project because of my int

Hamed Mohammadvand 7 Jun 10, 2022
Create VSCode Extensions with python

About Create vscode extensions with python. Installation Stable version: pip install vscode-ext Why use this? Why should you use this for building VSc

Swas.py 134 Jan 07, 2023
This is a Docker-based pipeline for preparing sextractor-ready multiwavelength images

Pipeline for creating NB422-detected (ODI) catalog The repository contains a Docker-based pipeline for preprocessing observational data. The pipeline

1 Sep 01, 2022
Hoopoe - Get notified of important stuff, right away.

Hoopoe - Get notified of important stuff, right away. Report a Bug · Request a Feature . Ask a Question Table of Contents About Getting Started Prereq

Vahid Al 8 Nov 12, 2022
Track testrail productivity in automated reporting to multiple teams

django_web_app_for_testrail testrail is a test case management tool which helps any organization to track all consumption and testing of manual and au

Vignesh 2 Nov 21, 2021
Snakemake worflow to process and filter long read data from Oxford Nanopore Technologies.

Nanopore-Workflow Snakemake workflow to process and filter long read data from Oxford Nanopore Technologies. It is designed to compare whole human gen

5 May 13, 2022
Taking the fight to the establishment.

Throwdown Taking the fight to the establishment. Wat? I wanted a simple markdown interpreter in python and/or javascript to output html for my website

Trevor van Hoof 1 Feb 01, 2022
A professional version for LBS

呐 Yuki Pro~ 懒兵服御用版本,yuki小姐觉得没必要单独造一个仓库,但懒兵觉得有必要并强制执行 将na-yuki框架抽象为模块,功能拆分为独立脚本,使用脚本注释器使其作为py运行 文件结构: na_yuki_pro_example.py 是一个说明脚本,用来直观展示na,yuki! Pro

1 Dec 21, 2021
Block fingerprinting for the beacon chain, for client identification & client diversity metrics

blockprint This is a repository for discussion and development of tools for Ethereum block fingerprinting. The primary aim is to measure beacon chain

Sigma Prime 49 Dec 08, 2022
The little-endian version of MessagePack

MessagePackEL This is the little-endian version of MessagePack, except the endianness is different, the rest is exactly the same as MessagePack. C lib

dukelec 9 May 13, 2022
The best free and open-source automated time tracker. Cross-platform, extensible, privacy-focused.

Records what you do so that you can know how you've spent your time. All in a secure way where you control the data. Website — Forum — Documentation —

ActivityWatch 7.8k Jan 09, 2023