A modular framework for vision & language multimodal research from Facebook AI Research (FAIR)

Last update: Jan 04, 2023

Overview

MMF is a modular framework for vision and language multimodal research from Facebook AI Research. MMF contains reference implementations of state-of-the-art vision and language models and has powered multiple research projects at Facebook AI Research. See full list of project inside or built on MMF here.

MMF is powered by PyTorch, allows distributed training and is un-opinionated, scalable and fast. Use MMF to bootstrap for your next vision and language multimodal research project by following the installation instructions. Take a look at list of MMF features here.

MMF also acts as starter codebase for challenges around vision and language datasets (The Hateful Memes, TextVQA, TextCaps and VQA challenges). MMF was formerly known as Pythia. The next video shows an overview of how datasets and models work inside MMF. Checkout MMF's video overview.

Installation

Follow installation instructions in the documentation.

Documentation

Learn more about MMF here.

Citation

If you use MMF in your work or use any models published in MMF, please cite:

@misc{singh2020mmf,
  author =       {Singh, Amanpreet and Goswami, Vedanuj and Natarajan, Vivek and Jiang, Yu and Chen, Xinlei and Shah, Meet and
                 Rohrbach, Marcus and Batra, Dhruv and Parikh, Devi},
  title =        {MMF: A multimodal framework for vision and language research},
  howpublished = {\url{https://github.com/facebookresearch/mmf}},
  year =         {2020}
}

License

MMF is licensed under BSD license available in LICENSE file

Comments

[feat] Add ViLT model
Stack from ghstack:

#1142

#1106

#1103

-> #1102

#1141

#1101

Add ViLT model and unit tests. ViLT model from megaVLT, using ViT from huggingface.

Differential Revision: D31117427
CLA Signed Merged
opened by Ryan-Qiyu-Jiang 49
[feat] Add ViLT image and text embeddings
Stack from ghstack:

#1142

#1106

#1103

#1102

#1141

-> #1101

Add ViT image and Bert text embedding encoders for ViLT model.

Differential Revision: D31117428
CLA Signed Merged
opened by Ryan-Qiyu-Jiang 46
[feat][PL][8/N] save config with checkpoint
Stack from ghstack:

#1035 [feat][PL][11/N] Inference with test

#1033 [feat][PL][10/N] lightning distributed

#1030 [refactor][PL][9/N] change the name of the _load_checkpoint to be more descriptive to contain test

#1024 [feat][PL][8/N] save config with checkpoint

#1023 [feat][PL][7/N] trainer checkpoint from mmf to lightning update

#1022 [feat][PL][6/N] pretrained state mapping for PL

Differential Revision: D29670421
CLA Signed Merged
opened by ytsheng 39
[feat][PL][6/N] pretrained state mapping for PL
Stack from ghstack:

#1035 [feat][PL][11/N] Inference with test

#1033 [feat][PL][10/N] lightning distributed

#1030 [refactor][PL][9/N] change the name of the _load_checkpoint to be more descriptive to contain test

#1024 [feat][PL][8/N] save config with checkpoint

#1023 [feat][PL][7/N] trainer checkpoint from mmf to lightning update

#1022 [feat][PL][6/N] pretrained state mapping for PL

Differential Revision: D29670417
CLA Signed
opened by ytsheng 37
[refactor][PL][9/N] change the name of the _load_checkpoint to be more descriptive to contain test
Stack from ghstack:

#1035 [feat][PL][11/N] Inference with test

#1033 [feat][PL][10/N] lightning distributed

#1030 [refactor][PL][9/N] change the name of the _load_checkpoint to be more descriptive to contain test

#1024 [feat][PL][8/N] save config with checkpoint

#1023 [feat][PL][7/N] trainer checkpoint from mmf to lightning update

#1022 [feat][PL][6/N] pretrained state mapping for PL

Differential Revision: D29888294
CLA Signed
opened by ytsheng 36
[feat] Add ViLT encoder
Stack from ghstack:

#1142

#1106

#1103

#1102

#1141

#1101

-> #1100

Add encoder for loading pretrained ViTModel weights from huggingface

Differential Revision: D31117425
CLA Signed
opened by Ryan-Qiyu-Jiang 33
[feat][PL][10/N] lightning distributed
Stack from ghstack:

#1035 [feat][PL][11/N] Inference with test

#1033 [feat][PL][10/N] lightning distributed

#1030 [refactor][PL][9/N] change the name of the _load_checkpoint to be more descriptive to contain test

#1024 [feat][PL][8/N] save config with checkpoint

#1023 [feat][PL][7/N] trainer checkpoint from mmf to lightning update

#1022 [feat][PL][6/N] pretrained state mapping for PL

Differential Revision: D29981634
CLA Signed
opened by ytsheng 33
[feat][PL][7/N] trainer checkpoint from mmf to lightning update
Stack from ghstack:

#1035 [feat][PL][11/N] Inference with test

#1033 [feat][PL][10/N] lightning distributed

#1030 [refactor][PL][9/N] change the name of the _load_checkpoint to be more descriptive to contain test

#1024 [feat][PL][8/N] save config with checkpoint

#1023 [feat][PL][7/N] trainer checkpoint from mmf to lightning update

#1022 [feat][PL][6/N] pretrained state mapping for PL

Differential Revision: D29670422
CLA Signed
opened by ytsheng 33
[feat] Add ViT modules
Stack from ghstack:

#1142

#1106

#1103

#1102

#1141

#1101

#1100

-> #1099

Add modified huggingface ViT modules from megaVLT supporting attention masks

Differential Revision: D31117429
CLA Signed
opened by Ryan-Qiyu-Jiang 30
[feat] Add UNITER model wrapper
Stack from ghstack:

#1144

#1128

-> #1127

#1133

Add UNITER model to mmf registry with support for pretraining through yaml head configs.

Differential Revision: D31768457
CLA Signed
opened by Ryan-Qiyu-Jiang 28
Error during training Textvqa

❓ Questions and Help

Hi,

While I am trying the training code with m4c model, I am getting the following error,

2021-03-11T03:34:15 | mmf.utils.general: Total Parameters: 90850184. Trained Parameters: 90850184 2021-03-11T03:34:15 | mmf.trainers.core.training_loop: Starting training... Traceback (most recent call last): File "C:\Users\kvman\anaconda3\envs\mmf\Scripts\mmf_run-script.py", line 33, in sys.exit(load_entry_point('mmf', 'console_scripts', 'mmf_run')()) File "d:\project\new folder\mmf\mmf_cli\run.py", line 133, in run main(configuration, predict=predict) File "d:\project\new folder\mmf\mmf_cli\run.py", line 56, in main trainer.train() File "d:\project\new folder\mmf\mmf\trainers\mmf_trainer.py", line 132, in train self.training_loop() File "d:\project\new folder\mmf\mmf\trainers\core\training_loop.py", line 31, in training_loop self.run_training_epoch() File "d:\project\new folder\mmf\mmf\trainers\core\training_loop.py", line 74, in run_training_epoch for idx, batch in enumerate(self.train_loader): File "C:\Users\kvman\anaconda3\envs\mmf\lib\site-packages\torch\utils\data\dataloader.py", line 363, in next data = self._next_data() File "C:\Users\kvman\anaconda3\envs\mmf\lib\site-packages\torch\utils\data\dataloader.py", line 989, in _next_data return self._process_data(data) File "C:\Users\kvman\anaconda3\envs\mmf\lib\site-packages\torch\utils\data\dataloader.py", line 1014, in _process_data data.reraise() File "C:\Users\kvman\anaconda3\envs\mmf\lib\site-packages\torch_utils.py", line 395, in reraise raise self.exc_type(msg) KeyError: Caught KeyError in DataLoader worker process 0. Original Traceback (most recent call last): File "d:\project\new folder\mmf\mmf\datasets\databases\readers\feature_readers.py", line 231, in load image_id = int(split.split("")[-1]) ValueError: invalid literal for int() with base 10: 'train\7f14a505b6edcbc5'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users\kvman\anaconda3\envs\mmf\lib\site-packages\torch\utils\data_utils\worker.py", line 185, in _worker_loop data = fetcher.fetch(index) File "C:\Users\kvman\anaconda3\envs\mmf\lib\site-packages\torch\utils\data_utils\fetch.py", line 44, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "C:\Users\kvman\anaconda3\envs\mmf\lib\site-packages\torch\utils\data_utils\fetch.py", line 44, in data = [self.dataset[idx] for idx in possibly_batched_index] File "C:\Users\kvman\anaconda3\envs\mmf\lib\site-packages\torch\utils\data\dataset.py", line 207, in getitem return self.datasets[dataset_idx][sample_idx] File "d:\project\new folder\mmf\mmf\datasets\builders\textvqa\dataset.py", line 100, in getitem features = self.features_db[idx] File "d:\project\new folder\mmf\mmf\datasets\databases\features_database.py", line 91, in getitem return self.get(image_info) File "d:\project\new folder\mmf\mmf\datasets\databases\features_database.py", line 99, in get return self.from_path(feature_path) File "d:\project\new folder\mmf\mmf\datasets\databases\features_database.py", line 107, in from_path features, infos = self._get_image_features_and_info(path) File "d:\project\new folder\mmf\mmf\datasets\databases\features_database.py", line 80, in _get_image_features_and_info image_feats, infos = self._read_features_and_info(feat_file) File "d:\project\new folder\mmf\mmf\datasets\databases\features_database.py", line 65, in _read_features_and_info feature, info = feature_reader.read(feat_file) File "d:\project\new folder\mmf\mmf\datasets\databases\readers\feature_readers.py", line 95, in read return self.feat_reader.read(image_feat_path) File "d:\project\new folder\mmf\mmf\datasets\databases\readers\feature_readers.py", line 158, in read image_info = self._load(image_feat_path) File "d:\project\new folder\mmf\mmf\datasets\databases\readers\feature_readers.py", line 238, in _load img_id_idx = self.image_id_indices[image_id] KeyError: b'train\7f14a505b6edcbc5'

When I tried with model = "Lorra", I am getting the below error,

2021-03-11T03:27:37 | mmf.utils.general: Total Parameters: 192497485. Trained Parameters: 192497485 2021-03-11T03:27:37 | mmf.trainers.core.training_loop: Starting training... Traceback (most recent call last): File "C:\Users\kvman\anaconda3\envs\mmf\Scripts\mmf_run-script.py", line 33, in sys.exit(load_entry_point('mmf', 'console_scripts', 'mmf_run')()) File "d:\project\new folder\mmf\mmf_cli\run.py", line 133, in run main(configuration, predict=predict) File "d:\project\new folder\mmf\mmf_cli\run.py", line 56, in main trainer.train() File "d:\project\new folder\mmf\mmf\trainers\mmf_trainer.py", line 132, in train self.training_loop() File "d:\project\new folder\mmf\mmf\trainers\core\training_loop.py", line 31, in training_loop self.run_training_epoch() File "d:\project\new folder\mmf\mmf\trainers\core\training_loop.py", line 74, in run_training_epoch for idx, batch in enumerate(self.train_loader): File "d:\project\new folder\mmf\mmf\datasets\multi_dataset_loader.py", line 213, in iter return iter(self.loaders[0]) File "C:\Users\kvman\anaconda3\envs\mmf\lib\site-packages\torch\utils\data\dataloader.py", line 291, in iter return _MultiProcessingDataLoaderIter(self) File "C:\Users\kvman\anaconda3\envs\mmf\lib\site-packages\torch\utils\data\dataloader.py", line 737, in init w.start() File "C:\Users\kvman\anaconda3\envs\mmf\lib\multiprocessing\process.py", line 112, in start self._popen = self._Popen(self) File "C:\Users\kvman\anaconda3\envs\mmf\lib\multiprocessing\context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "C:\Users\kvman\anaconda3\envs\mmf\lib\multiprocessing\context.py", line 322, in _Popen return Popen(process_obj) File "C:\Users\kvman\anaconda3\envs\mmf\lib\multiprocessing\popen_spawn_win32.py", line 89, in init reduction.dump(process_obj, to_child) File "C:\Users\kvman\anaconda3\envs\mmf\lib\multiprocessing\reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) BrokenPipeError: [Errno 32] Broken pipe

Kindly help me to resolve this issue.
needs more info triaged

opened by Mano2610 28

$errors while running pytests .\test$

errors while running pytests .\test

❓ Questions and Help

I successfully installed mmf according to the installation document.

conda create -n mmf python=3.7
conda activate mmf
git clone https://github.com/facebookresearch/mmf.git
cd mmf
pip install --editable .

but, in test step, I got below error messages. I goggled about libcudart.so and found out that it has to do with torchaudio. As reqirement.txt, my torchaudio version is 0.11.0. Anyone who knows to solve this problem?

Thank you very much. Minho

$ pytest ./tests/
ImportError while loading conftest '/home/mhlee/mmf/tests/conftest.py'.
tests/__init__.py:2: in <module>
    from mmf.utils.patch import patch_transformers
mmf/__init__.py:6: in <module>
    patch_transformers()
mmf/utils/patch.py:65: in patch_transformers
    f"transformers.models.{key}.{module}"
../.local/lib/python3.7/site-packages/transformers/models/speech_to_text/feature_extraction_speech_to_text.py:23: in <module>
    import torchaudio.compliance.kaldi as ta_kaldi
../.local/lib/python3.7/site-packages/torchaudio/__init__.py:1: in <module>
    from torchaudio import _extension  # noqa: F401
../.local/lib/python3.7/site-packages/torchaudio/_extension.py:67: in <module>
    _init_extension()
../.local/lib/python3.7/site-packages/torchaudio/_extension.py:61: in _init_extension
    _load_lib("libtorchaudio")
../.local/lib/python3.7/site-packages/torchaudio/_extension.py:51: in _load_lib
    torch.ops.load_library(path)
../.local/lib/python3.7/site-packages/torch/_ops.py:220: in load_library
    ctypes.CDLL(path)
/home/master/anaconda3/envs/mmf/lib/python3.7/ctypes/__init__.py:364: in __init__
    self._handle = _dlopen(self._name, mode)
E   OSError: libcudart.so.10.2: cannot open shared object file: No such file or directory

opened by cokemhlee 0

Bump terser from 5.10.0 to 5.16.1 in /website
Bumps terser from 5.10.0 to 5.16.1.

Changelog

Sourced from terser's changelog.

v5.16.1

Properly handle references in destructurings (const { [reference]: val } = ...)

Allow parsing of .#privatefield in nested classes

Do not evaluate operations that return large strings if that would make the output code larger

Make collapse_vars handle block scope correctly

Internal improvements: Typos (#1311), more tests, small-scale refactoring

v5.16.0

Disallow private fields in object bodies (#1011)

Parse #privatefield in object (#1279)

Compress #privatefield in object

v5.15.1

Fixed missing parentheses around optional chains

Avoid bare let or const as the bodies of if statements (#1253)

Small internal fixes (#1271)

Avoid inlining a class twice and creating two equivalent but !== classes.

v5.15.0

Basic support for ES2022 class static initializer blocks.

Add AudioWorkletNode constructor options to domprops list (#1230)

Make identity function inliner not inline id(...expandedArgs)

v5.14.2

Security fix for RegExps that should not be evaluated (regexp DDOS)

Source maps improvements (#1211)

Performance improvements in long property access evaluation (#1213)

v5.14.1

keep_numbers option added to TypeScript defs (#1208)

Fixed parsing of nested template strings (#1204)

v5.14.0

Switched to @jridgewell/source-map for sourcemap generation (#1190, #1181)

Fixed source maps with non-terminated segments (#1106)

Enabled typescript types to be imported from the package (#1194)

Extra DOM props have been added (#1191)

Delete the AST while generating code, as a means to save RAM

v5.13.1

Removed self-assignments (varname=varname) (closes #1081)

Separated inlining code (for inlining things into references, or removing IIFEs)

Allow multiple identifiers with the same name in var destructuring (eg var { a, a } = x) (#1176)

v5.13.0

... (truncated)

Commits

b90215a 5.16.1

319fe31 update changelog

d941a11 mark computed properties in for..of initializers as used. Closes #1238

635400e move AST_Scope#drop_unused into its own file

a192fac some cleanups

dcc3030 add shadowed variable test

4d37f39 traverse exported code while hoisting object properties. Closes #1307

1e99e5d update test

68461d4 properly detect nested classes. Closes #1313

b91e0ce docs: fix simple typo, concatentation -> concatenation (#1311)

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

CLA Signed dependencies javascript
opened by dependabot[bot] 0
Bump minimist from 1.2.5 to 1.2.7 in /website
Bumps minimist from 1.2.5 to 1.2.7.

Changelog

Sourced from minimist's changelog.

v1.2.7 - 2022-10-10

Commits

[meta] add auto-changelog 0ebf4eb

[actions] add reusable workflows e115b63

[eslint] add eslint; rules to enable later are warnings f58745b

[Dev Deps] switch from covert to nyc ab03356

[readme] rename and add badges 236f4a0

[meta] create FUNDING.yml; add funding in package.json 783a49b

[meta] use npmignore to autogenerate an npmignore file f81ece6

Only apps should have lockfiles 56cad44

[Dev Deps] update covert, tape; remove unnecessary tap 49c5f9f

[Tests] add aud in posttest 228ae93

[meta] add safe-publish-latest 01fc23f

[meta] update repo URLs 6b164c7

v1.2.6 - 2022-03-21

Commits

test from prototype pollution PR bc8ecee

isConstructorOrProto adapted from PR c2b9819

security notice for additional prototype pollution issue ef88b93

Commits

c590d75 v1.2.7

0ebf4eb [meta] add auto-changelog

e115b63 [actions] add reusable workflows

01fc23f [meta] add safe-publish-latest

f58745b [eslint] add eslint; rules to enable later are warnings

228ae93 [Tests] add aud in posttest

236f4a0 [readme] rename and add badges

ab03356 [Dev Deps] switch from covert to nyc

49c5f9f [Dev Deps] update covert, tape; remove unnecessary tap

783a49b [meta] create FUNDING.yml; add funding in package.json

Additional commits viewable in compare view

Maintainer changes

This version was pushed to npm by ljharb, a new releaser for minimist since your current version.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

CLA Signed dependencies javascript
opened by dependabot[bot] 0
Bump prismjs from 1.26.0 to 1.29.0 in /website
Bumps prismjs from 1.26.0 to 1.29.0.

Release notes

Sourced from prismjs's releases.

v1.29.0

Release 1.29.0

v1.28.0

Release 1.28.0

v1.27.0

Release 1.27.0

Changelog

Sourced from prismjs's changelog.

1.29.0 (2022-08-23)

New components

BBj (#3511) 1134bdfc

BQN (#3515) 859f99a0

Cilk/C & Cilk/C++ (#3522) c8462a29

Gradle (#3443) 32119823

METAFONT (#3465) 2815f699

WGSL (#3455) 4c87d418

Updated components

AsciiDoc

Some regexes are too greedy (#3481) c4cbeeaa

Bash

Added "sh" alias (#3509) 6b824d47

Added support for parameters and the java and sysctl commands. (#3505) b9512b22

Added cargo command (#3488) 3e937137

BBj

Improve regexes (#3512) 0cad9ae5

CSS

Fixed @-rules not accounting for strings (#3438) 0d4b6cb6

CSS Extras

Added support for RebeccaPurple color (#3448) 646b2e0a

Hoon

Fixed escaped strings (#3473) 64642716

Java

Added support for constants (#3507) 342a0039

Markup

Fixed quotes in HTML attribute values (#3442) ca8eaeee

NSIS

Added missing commands (#3504) b0c2a9b4

Scala

Updated keywords to support Scala 3 (#3506) a090d063

SCSS

Fix casing in title of the scss lang (#3501) 2aed9ce7

Updated plugins

Line Highlight

Account for offset when clamping ranges (#3518) 098e3000

Ignore ranges outside of actual lines (#3475) 9a4e725b

Normalize Whitespace

Add configuration via attributes (#3467) 91dea0c8

Other

Added security policy (#3070) 05ee042a

Added list of maintainers (#3410) 866b302e

... (truncated)

Commits

59e5a34 1.29.0

cd080f2 Updated npmignore to include new MD files (#3534)

751664b Added PR stop notice (#3532)

248f6ab Added changelog for v1.29.0 (#3533)

098e300 Line Highlight: Account for offset when clamping ranges (#3518)

6b824d4 Bash: Added "sh" alias (#3509)

15272f7 Website: Added third-party tutorial for Pug template (#3459)

c8462a2 Cilk: Add support for Cilk (with C/C++) (#3522)

859f99a Added bqn language support (#3515)

0cad9ae BBj: Improve regexes (#3512)

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

CLA Signed dependencies javascript
opened by dependabot[bot] 0
Bump eventsource from 1.1.0 to 1.1.2 in /website
Bumps eventsource from 1.1.0 to 1.1.2.

Changelog

Sourced from eventsource's changelog.

1.1.2

Inline origin resolution, drops original dependency (#281 Espen Hovlandsdal)

1.1.1

Do not include authorization and cookie headers on redirect to different origin (#273 Espen Hovlandsdal)

Commits

0a8b85b 1.1.2

f99ae66 docs: update history for 1.1.2

06c9721 chore: rebuild polyfill

9494642 fix: inline origin resolution, drop original dependency (#281)

aa7a408 1.1.1

56d489e chore: rebuild polyfill

4a951e5 docs: update history for 1.1.1

f9f6416 fix: strip sensitive headers on redirect to different origin

See full diff in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

CLA Signed dependencies javascript
opened by dependabot[bot] 0
Bump follow-redirects from 1.14.7 to 1.15.2 in /website
Bumps follow-redirects from 1.14.7 to 1.15.2.

Commits

9655237 Release version 1.15.2 of the npm package.

6e2b86d Default to localhost if no host given.

449e895 Throw invalid URL error on relative URLs.

e30137c Use type functions.

76ea31f ternary operator syntax fix

84c00b0 HTTP header lines are separated by CRLF.

d28bcbf Create SECURITY.md (#202)

62a551c Release version 1.15.1 of the npm package.

7fe0779 Use for ... of.

948c30c Fix redirecting to relative URL when using proxy

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

CLA Signed dependencies javascript
opened by dependabot[bot] 0

Releases(v0.3.1)

v0.3.1(Aug 26, 2019)

Source code(tar.gz)
Source code(zip)
v0.3(Jun 26, 2019)
Features

Multi-tasking support: Multitasking over various datasets available in Pythia

Distributed training support

Better Customization Support: Use your custom losses, metrics, optimizers and lot of more

Standardized Trainer API: A standard trainer API to fit most of your use-cases, if not inherit and build your own trainer

Processors: Use processors to build out your datasets easily and without pain

SampleList and Sample: Use SampleList and Sample to have more granular control over what you pass and a single unified API for accessing attributes whether inside a dataset, a single sample or a batch

Feature Extraction: A new simple script to extract out features and related information from VQA MaskRCNN Benchmark

Registry: No need to manually load datasets and models anymore, registry takes care of loading your models, datasets and other classes at the fly. Think of registry as a singleton containing all that you need.

Tensorboard Logging: Tensorboard logging is now provided by default.

Configuration: Better hierarchal configuration system for better separation of concerns.

Checkpointing: Better control checkpointing and resuming

Logging: Better logging is provided now with eta, individual val losses and metrics. Just pass them back from your model and everything logs automatically

EvalAI Evaluation: Now, directly output JSON files that can be uploaded to EvalAI

Early Stopping

New Embeddings Support: FastText, GloVe, BERT etc.

Datasets

Following new datasets were added:

TextVQA

VizWiz

COCO Captioning

Preliminary GQA support

Models

LoRRA

Revamped Pythia

BUTD

Preliminary BAN Support

Note: There are a lot of breaking changes in the API from v0.1. Refer to the documentation to learn more on how to work with Pythia v0.3.
Source code(tar.gz)
Source code(zip)

Owner

Facebook Research

GitHub Repository https://mmf.sh/

COLMAP - Structure-from-Motion and Multi-View Stereo

COLMAP About COLMAP is a general-purpose Structure-from-Motion (SfM) and Multi-View Stereo (MVS) pipeline with a graphical and command-line interface.

4.7k Jan 07, 2023

USAD - UnSupervised Anomaly Detection on multivariate time series

USAD - UnSupervised Anomaly Detection on multivariate time series Scripts and utility programs for implementing the USAD architecture. Implementation

116 Jan 04, 2023

Pre-trained BERT Models for Ancient and Medieval Greek, and associated code for LaTeCH 2021 paper titled - "A Pilot Study for BERT Language Modelling and Morphological Analysis for Ancient and Medieval Greek"

Ancient Greek BERT The first and only available Ancient Greek sub-word BERT model! State-of-the-art post fine-tuning on Part-of-Speech Tagging and Mor

22 Dec 08, 2022

(ICCV 2021) ProHMR - Probabilistic Modeling for Human Mesh Recovery

ProHMR - Probabilistic Modeling for Human Mesh Recovery Code repository for the paper: Probabilistic Modeling for Human Mesh Recovery Nikos Kolotouros

209 Dec 13, 2022

Apache Flink

Apache Flink Apache Flink is an open source stream processing framework with powerful stream- and batch-processing capabilities. Learn more about Flin

20.4k Dec 30, 2022

Official implementation of "Synthetic Temporal Anomaly Guided End-to-End Video Anomaly Detection" (ICCV Workshops 2021: RSL-CV).

Official PyTorch implementation of "Synthetic Temporal Anomaly Guided End-to-End Video Anomaly Detection" This is the implementation of the paper "Syn

11 Oct 07, 2022

custom pytorch implementation of MoCo v3

MoCov3-pytorch custom implementation of MoCov3 [arxiv]. I made minor modifications based on the official MoCo repository [github]. No ViT part code an

39 Nov 14, 2022

Official code for the ICCV 2021 paper "DECA: Deep viewpoint-Equivariant human pose estimation using Capsule Autoencoders"

DECA Official code for the ICCV 2021 paper "DECA: Deep viewpoint-Equivariant human pose estimation using Capsule Autoencoders". All the code is writte

23 Dec 01, 2022

Code of TVT: Transferable Vision Transformer for Unsupervised Domain Adaptation

TVT Code of TVT: Transferable Vision Transformer for Unsupervised Domain Adaptation Datasets: Digit: MNIST, SVHN, USPS Object: Office, Office-Home, Vi

37 Dec 15, 2022

[ICRA 2022] An opensource framework for cooperative detection. Official implementation for OPV2V.

OpenCOOD OpenCOOD is an Open COOperative Detection framework for autonomous driving. It is also the official implementation of the ICRA 2022 paper OPV

322 Dec 23, 2022

xitorch: differentiable scientific computing library

xitorch is a PyTorch-based library of differentiable functions and functionals that can be widely used in scientific computing applications as well as deep learning.

24 Apr 15, 2021

A pytorch-based real-time segmentation model for autonomous driving

CFPNet: Channel-Wise Feature Pyramid for Real-Time Semantic Segmentation This project contains the Pytorch implementation for the proposed CFPNet: pap

342 Dec 22, 2022

Tensorflow Implementation of ECCV'18 paper: Multimodal Human Motion Synthesis

MT-VAE for Multimodal Human Motion Synthesis This is the code for ECCV 2018 paper MT-VAE: Learning Motion Transformations to Generate Multimodal Human

36 Oct 02, 2022

Using the provided dataset which includes various book features, in order to predict the price of books, using various proposed methods and models.

1 Jan 13, 2022

Deep Q-network learning to play flappybird.

AI Plays Flappy Bird I've trained a DQN that learns to play flappy bird on it's own. Try the pre-trained model First install the pip requirements and

3 Mar 01, 2022

This is a custom made virus code in python, using tkinter module.

skeleterrorBetaV0.1-Virus-code This is a custom made virus code in python, using tkinter module. This virus is not harmful to the computer, it only ma

0 Nov 21, 2022

Implementation of DropLoss for Long-Tail Instance Segmentation in Pytorch

[AAAI 2021]DropLoss for Long-Tail Instance Segmentation [AAAI 2021] DropLoss for Long-Tail Instance Segmentation Ting-I Hsieh*, Esther Robb*, Hwann-Tz

37 Dec 02, 2022

Implementation for "Seamless Manga Inpainting with Semantics Awareness" (SIGGRAPH 2021 issue)

Seamless Manga Inpainting with Semantics Awareness [SIGGRAPH 2021](To appear) | Project Website | BibTex Introduction: Manga inpainting fills up the d

101 Jan 01, 2023

3D AffordanceNet is a 3D point cloud benchmark consisting of 23k shapes from 23 semantic object categories, annotated with 56k affordance annotations and covering 18 visual affordance categories.

3D AffordanceNet This repository is the official experiment implementation of 3D AffordanceNet benchmark. 3D AffordanceNet is a 3D point cloud benchma

49 Dec 01, 2022

[CVPR'22] Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast

wseg Overview The Pytorch implementation of Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast. [arXiv] Though image-level weakly

96 Dec 30, 2022

A modular framework for vision & language multimodal research from Facebook AI Research (FAIR)

Related tags

Overview

Installation

Documentation

Citation

License

Comments

❓ Questions and Help

❓ Questions and Help

v5.16.1

v5.16.0

v5.15.1

v5.15.0

v5.14.2

v5.14.1

v5.14.0

v5.13.1

v5.13.0

v1.2.7 - 2022-10-10

Commits

v1.2.6 - 2022-03-21

Commits

v1.29.0

v1.28.0

v1.27.0

1.29.0 (2022-08-23)

New components

Updated components

Updated plugins

Other

Releases(v0.3.1)

v0.3.1(Aug 26, 2019)

v0.3(Jun 26, 2019)

Features

Datasets

Models

Owner

Facebook Research

COLMAP - Structure-from-Motion and Multi-View Stereo

USAD - UnSupervised Anomaly Detection on multivariate time series

Pre-trained BERT Models for Ancient and Medieval Greek, and associated code for LaTeCH 2021 paper titled - "A Pilot Study for BERT Language Modelling and Morphological Analysis for Ancient and Medieval Greek"

(ICCV 2021) ProHMR - Probabilistic Modeling for Human Mesh Recovery

Apache Flink

Official implementation of "Synthetic Temporal Anomaly Guided End-to-End Video Anomaly Detection" (ICCV Workshops 2021: RSL-CV).

custom pytorch implementation of MoCo v3

Official code for the ICCV 2021 paper "DECA: Deep viewpoint-Equivariant human pose estimation using Capsule Autoencoders"

Code of TVT: Transferable Vision Transformer for Unsupervised Domain Adaptation

[ICRA 2022] An opensource framework for cooperative detection. Official implementation for OPV2V.

xitorch: differentiable scientific computing library

A pytorch-based real-time segmentation model for autonomous driving

Tensorflow Implementation of ECCV'18 paper: Multimodal Human Motion Synthesis

Using the provided dataset which includes various book features, in order to predict the price of books, using various proposed methods and models.

Deep Q-network learning to play flappybird.

This is a custom made virus code in python, using tkinter module.

Implementation of DropLoss for Long-Tail Instance Segmentation in Pytorch

Implementation for "Seamless Manga Inpainting with Semantics Awareness" (SIGGRAPH 2021 issue)

3D AffordanceNet is a 3D point cloud benchmark consisting of 23k shapes from 23 semantic object categories, annotated with 56k affordance annotations and covering 18 visual affordance categories.

[CVPR'22] Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast