fds is a tool for Data Scientists made by DAGsHub to version control data and code at once.

Overview

Fast Data Science aka fds

Discord Tests PyPI MIT license


fds is a tool for Data Scientists made by DAGsHub to version control data and code at once.

At a high level, fds is a command line wrapper around Git and DVC, meant to minimize the chances of human error, automate repetitive tasks, and provide a smoother landing for new users.

See the launch blog for more information about the motivation behind this project.

Installation

  • Install fds using PIP pip install fastds
  • Once installed successfully, you can start using fds
  • eg: fds init should trigger the init command
  • You can also use sdf instead of fds - it's identical, but might be more fun to type 🤓

Commands Supported

$ fds -h
usage: fds [-h] [-v] {init,status,add,commit,push,save} ...

One command for all your git and dvc needs

positional arguments:
  {init,status,add,commit,push,save}
                        command (refer commands section in documentation)
    init                initialize a git and dvc repository
    status              get status of your git and dvc repository
    add                 add files/folders to git and dvc repository
    commit              commits added changes to git and dvc repository
    push                push commits to remote git and dvc repository
    save                saves all project files to a new version and pushes
                        them to your remote

Examples

fds status = dvc status + git status

fds status lets us quickly check the full status of the repo - both DVC and git at the same time, to make sure we don't forget anything.

image

Here, we can see that we have a small, normal text file - .gitignore, plus a bigfile.txt and data folder which we would want to add to DVC and not to git. fds add makes that easy!

fds add = dvc add + git add wizard 🧙‍♂️

You're probably used to the convenience of using git add . to just track everything. Unfortunately, you have to be careful doing this when working with large files - one wrong move, and you might fry your hard drive by accidentally telling git to track a huge dataset!
We wanted to retain the convenience of just typing one command which means "just track all changes, I'll do a git commit in one second", which will be smart enough to avoid the pitfalls of large data files.
fds add does exactly that, while interactively asking the user how to handle files. You can add to DVC, or git, recursively step into large folders, skip or ignore files, etc.

image

Here's the file tree of the repo I used above, with file sizes included. Note how bigfile.txt and data/ were automatically added to DVC and not git:

image

fds commit = dvc commit + git commit

Finally, to close the loop of a real workflow, what happens when I change existing DVC tracked files? Without FDS, you'd have to remember to separately run dvc repro or dvc commit, then git add tracked_file.dvc, and only then git commit.
fds commit does all that for you - commits changes to DVC first, then adds the .dvc files with the updated hashes to git, then immediately commits these changes (plus any other staged changes) to a new git commit. Voila!

image

Contributing

We would love for you to try out FDS yourself, and to give us feedback. It would really help us to prioritize future features, so please vote on or create issues!
If you'd like to take a more active part, we have some good first issues that you can start with. We'll be happy to provide guidance on the best way to do so.

And of course, we're always happy to have you on the DAGsHub discord, where you can ask questions or give feedback on FDS: Discord


Made with ❤️   by dagshub-logo
Comments
  • Check latest version on PyPI and suggest to upgrade

    Check latest version on PyPI and suggest to upgrade

    On any command of fds, it should check if a new version is available and either display a message clearly suggesting to upgrade, or actually ask whether to upgrade interactively. Not sure if the interactive upgrade is actually possible though.

    enhancement good first issue ux 
    opened by guysmoilov 10
  • fds init

    fds init

    I successfully pip installed fastds, but anytime I did fds init, it always gives this import output after displaying some Traceback information: ImportError: cannot import name '__version__' from 'fds.__init__' < c:\users\<MyComputerName>\pneumonia-detection\.venv\lib\site-packages\fds\__init__.py>

    bug 
    opened by Chibuzo75 9
  • Fail tests on flake8 failure

    Fail tests on flake8 failure

    For syntax errors we anyways fail the test, for the other errors we show them as warnings, but if the total count of these exceeds 10 then we throw an error asking user to fix those lint errors

    tests 
    opened by mohithg 5
  • Documentation request: using fds with git repo already using dvc

    Documentation request: using fds with git repo already using dvc

    It's not clear from the readme, or the blog posting, how to start using fds with a git repo already using dvc.

    • https://github.com/DAGsHub/fds/blob/main/README.md
    • https://dagshub.com/blog/fds-fast-data-science-with-git-and-dvc/

    I can guess that all I need to do is install the fds python package, and I use pipenv, so that might work, but it isn't clear. I know "make a copy of your repo and try it out!" but how many experiments until I recreate the documentation that is so badly needed?

    documentation 
    opened by drjasonharrison 4
  • feat(run): implement pre-command hook runner to handle edge cases

    feat(run): implement pre-command hook runner to handle edge cases

    • New HooksRunner cls to contain pre-command hooks
    • Move existing hooks+logic into new class
    • Automatically trigger (with user confirmation) git init / dvc init

    fixes #35

    enhancement 
    opened by pwoolvett 4
  • Feature to check for latest version before running a command

    Feature to check for latest version before running a command

    With the package version info from PyPi, fastds will now compare the local version with the latest version available on PyPI, to prompt for an upgrade if the local version is not the latest.

    enhancement ux 
    opened by indweller 4
  • fds clone need extra argument folder location

    fds clone need extra argument folder location

    For Git you can simply set the location of local repository. For example

    git clone <remote> <local directory>
    
    

    It is not urgent issue, but in annoyance category.

    ux 
    opened by kingabzpro 2
  • Auto update FDS only tries with `pip` and not `pip3`

    Auto update FDS only tries with `pip` and not `pip3`

    fds asks if I want to update to the latest version, but when I type y to perform the update, I get thrown a bad interpreter error, due to it using pip which doesn't work on my system for a few reasons.

    It would be great if fds could check whether pip or pip3 work for the update, and use whichever one works.

    enhancement 
    opened by deanp70 2
  • Add `-y` option to let user decide to auto confirm all changed files

    Add `-y` option to let user decide to auto confirm all changed files

    As a side note, I think the way I added the argument makes more sense compared to how the --verbose flag was added, since it would currently accept an argument, which might cause issues. I haven't checked this, but @mohithg or @guysmoilov let me know if I'm missing something or we should change it too.

    enhancement 
    opened by deanp70 2
  • fds not working on Mac with whitespaces in the path

    fds not working on Mac with whitespaces in the path

    The path to my venv is /Users/deanpleban/Project Talos/Code/fds/env/bin/fds and whenever I run any fds command I get:

    /Users/deanpleban/Project Talos/Code/fds/env/bin/fds: bad interpreter: "/Users/deanpleban/Project: no such file or directory
    

    I tried googling around, but so far I can't find an explanation for how to solve this.

    opened by deanp70 2
  • Update pytest requirement from ~=6.2.5 to ~=7.1.3

    Update pytest requirement from ~=6.2.5 to ~=7.1.3

    Updates the requirements on pytest to permit the latest version.

    Release notes

    Sourced from pytest's releases.

    7.1.3

    pytest 7.1.3 (2022-08-31)

    Bug Fixes

    • #10060: When running with --pdb, TestCase.tearDown is no longer called for tests when the class has been skipped via unittest.skip or pytest.mark.skip.
    • #10190: Invalid XML characters in setup or teardown error messages are now properly escaped for JUnit XML reports.
    • #10230: Ignore .py files created by pyproject.toml-based editable builds introduced in pip 21.3.
    • #3396: Doctests now respect the --import-mode flag.
    • #9514: Type-annotate FixtureRequest.param as Any as a stop gap measure until 8073{.interpreted-text role="issue"} is fixed.
    • #9791: Fixed a path handling code in rewrite.py that seems to work fine, but was incorrect and fails in some systems.
    • #9917: Fixed string representation for pytest.approx{.interpreted-text role="func"} when used to compare tuples.

    Improved Documentation

    • #9937: Explicit note that tmpdir{.interpreted-text role="fixture"} fixture is discouraged in favour of tmp_path{.interpreted-text role="fixture"}.

    Trivial/Internal Changes

    Commits
    • 4645bcd Remove incorrect output in how-to/fixtures.rst
    • fadfb4f Prepare release version 7.1.3
    • ab96ea8 Merge pull request #10258 from pytest-dev/backport-10252-to-7.1.x
    • fc0e024 [7.1.x] Fix regendoc
    • 8f5088f Merge pull request #10249 from pytest-dev/backport-10231-to-7.1.x
    • aae93d6 Ignore type-errors related to attr.asdict
    • 71b79fc [7.1.x] Ignore editable installation modules
    • 89f7518 Merge pull request #10222 from pytest-dev/backport-10171-to-7.1.x
    • 88fc45b [7.1.x] Update fixtures.rst w/ finalizer order
    • d0b53d6 Merge pull request #10221 from pytest-dev/backport-10217-to-7.1.x
    • Additional commits viewable in compare view

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 1
  • Update pytest requirement from ~=6.2.5 to ~=7.2.0

    Update pytest requirement from ~=6.2.5 to ~=7.2.0

    Updates the requirements on pytest to permit the latest version.

    Release notes

    Sourced from pytest's releases.

    7.2.0

    pytest 7.2.0 (2022-10-23)

    Deprecations

    • #10012: Update pytest.PytestUnhandledCoroutineWarning{.interpreted-text role="class"} to a deprecation; it will raise an error in pytest 8.

    • #10396: pytest no longer depends on the py library. pytest provides a vendored copy of py.error and py.path modules but will use the py library if it is installed. If you need other py.* modules, continue to install the deprecated py library separately, otherwise it can usually be removed as a dependency.

    • #4562: Deprecate configuring hook specs/impls using attributes/marks.

      Instead use :pypytest.hookimpl{.interpreted-text role="func"} and :pypytest.hookspec{.interpreted-text role="func"}. For more details, see the docs <legacy-path-hooks-deprecated>{.interpreted-text role="ref"}.

    • #9886: The functionality for running tests written for nose has been officially deprecated.

      This includes:

      • Plain setup and teardown functions and methods: this might catch users by surprise, as setup() and teardown() are not pytest idioms, but part of the nose support.
      • Setup/teardown using the @​with_setup decorator.

      For more details, consult the deprecation docs <nose-deprecation>{.interpreted-text role="ref"}.

    Features

    • #9897: Added shell-style wildcard support to testpaths.

    Improvements

    • #10218: @pytest.mark.parametrize() (and similar functions) now accepts any Sequence[str] for the argument names, instead of just list[str] and tuple[str, ...].

      (Note that str, which is itself a Sequence[str], is still treated as a comma-delimited name list, as before).

    • #10381: The --no-showlocals flag has been added. This can be passed directly to tests to override --showlocals declared through addopts.

    • #3426: Assertion failures with strings in NFC and NFD forms that normalize to the same string now have a dedicated error message detailing the issue, and their utf-8 representation is expresed instead.

    • #7337: A warning is now emitted if a test function returns something other than [None]{.title-ref}. This prevents a common mistake among beginners that expect that returning a [bool]{.title-ref} (for example [return foo(a, b) == result]{.title-ref}) would cause a test to pass or fail, instead of using [assert]{.title-ref}.

    • #8508: Introduce multiline display for warning matching via :pypytest.warns{.interpreted-text role="func"} and enhance match comparison for :py_pytest._code.ExceptionInfo.match{.interpreted-text role="func"} as returned by :pypytest.raises{.interpreted-text role="func"}.

    • #8646: Improve :pypytest.raises{.interpreted-text role="func"}. Previously passing an empty tuple would give a confusing error. We now raise immediately with a more helpful message.

    • #9741: On Python 3.11, use the standard library's tomllib{.interpreted-text role="mod"} to parse TOML.

      tomli{.interpreted-text role="mod"}` is no longer a dependency on Python 3.11.

    • #9742: Display assertion message without escaped newline characters with -vv.

    • #9823: Improved error message that is shown when no collector is found for a given file.

    ... (truncated)

    Commits
    • 3af3f56 Prepare release version 7.2.0
    • bc2c3b6 Merge pull request #10408 from NateMeyvis/patch-2
    • d84ed48 Merge pull request #10409 from pytest-dev/asottile-patch-1
    • ffe49ac Merge pull request #10396 from pytest-dev/pylib-hax
    • d352098 allow jobs to pass if codecov.io fails
    • c5c562b Fix typos in CONTRIBUTING.rst
    • d543a45 add deprecation changelog for py library vendoring
    • f341a5c Merge pull request #10407 from NateMeyvis/patch-1
    • 1027dc8 [pre-commit.ci] auto fixes from pre-commit.com hooks
    • 6b905ee Add note on tags to CONTRIBUTING.rst
    • Additional commits viewable in compare view

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 0
  • Bump colorama from 0.4.4 to 0.4.6

    Bump colorama from 0.4.4 to 0.4.6

    Bumps colorama from 0.4.4 to 0.4.6.

    Changelog

    Sourced from colorama's changelog.

    0.4.6 Current release

    • tartley/colorama#139 Add alternative to 'init()', called 'just_fix_windows_console'. This fixes many longstanding problems with 'init', such as working incorrectly on modern Windows terminals, and wonkiness when init gets called multiple times. The intention is that it just makes all Windows terminals treat ANSI the same way as other terminals do. Many thanks the njsmith for fixing our messes.
    • tartley/colorama#352 Support Windows 10's ANSI/VT console. This didn't exist when Colorama was created, and avoiding us causing havok there is long overdue. Thanks to segeviner for the initial approach, and to njsmith for getting it merged.
    • tartley/colorama#338 Internal overhaul of package metadata declaration, which abolishes our use of the now heavily discouraged setuptools (and hence setup.py, setup.cfg and MANIFEST.in), in favor of hatchling (and hence pyproject.toml), generously contributed by ofek (author of hatchling). This includes dropping support Python3.5 and 3.6, which are EOL, and were already dropped from setuptools, so this should not affect our users.
    • tartley/colorama#353 Attention to detail award to LqdBcnAtWork for a spelling fix in demo06 0.4.5
    • Catch a racy ValueError that could occur on exit.
    • Create README-hacking.md, for Colorama contributors.
    • Tweak some README unicode characters that don't render correctly on PyPI.
    • Fix some tests that were failing on some operating systems.
    • Add support for Python 3.9.
    • Add support for PyPy3.
    • Add support for pickling with the dill module.
    Commits
    • 3de9f01 bump version 0.4.6
    • a45949b Format the CHANGELOG bullet list, no content change
    • f55f72e comment need for a fix after recent MP broke it
    • cb83041 fix test-release after recent MPs broke it
    • 832f14c README tweaks
    • 54b89c3 Bump version to 0.4.6rc1
    • 7991d34 'make bootstrap' uses system python3, not pinned v3.8
    • 0ae5ef2 CHANGELOG updates for upcoming 0.4.6 release
    • 52f4cfd Tweak ordering of release checklist
    • ab64cfa Merge pull request #353 from LqdBcnAtWork/patch-1
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 0
  • Bump humanize from 3.13.1 to 4.4.0

    Bump humanize from 3.13.1 to 4.4.0

    Bumps humanize from 3.13.1 to 4.4.0.

    Release notes

    Sourced from humanize's releases.

    4.4.0

    Added

    Fixed

    4.3.0

    Added

    Fixed

    4.2.3

    Fixed

    • Update annotations, docs, and tests: naturaltime can also accept a timedelta (#31) @​nuztalgia

    4.2.2

    Fixed

    • Update annotations: naturadelta and naturaltime can also accept a float (#29) @​hugovk

    4.2.1

    Fixed

    4.2.0

    Added

    Fixed

    ... (truncated)

    Commits
    • 5553ff5 Update FUNDING.yml
    • 694a2d6 Merge pull request #56 from python-humanize/add-installation-instructions
    • c0591ae Add installation instructions
    • e0e82da Merge pull request #54 from mjmikulski/pl-short-scale
    • a52beed Add author notice
    • 7b0d9be Fix wrong declinations of googol and a year
    • d0dde8f Replace short scale with long scale for Polish
    • 72d3ef4 Merge pull request #53 from Luflosi/implement-decimal-sep-i18n
    • 862748a Internationalise the decimal separator in intcomma()
    • d8e2739 Merge pull request #52 from Luflosi/fix-intcomma-with-str-and-ndigits
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 0
  • Bump requests from 2.26.0 to 2.28.1

    Bump requests from 2.26.0 to 2.28.1

    Bumps requests from 2.26.0 to 2.28.1.

    Release notes

    Sourced from requests's releases.

    v2.28.1

    2.28.1 (2022-06-29)

    Improvements

    • Speed optimization in iter_content with transition to yield from. (#6170)

    Dependencies

    • Added support for chardet 5.0.0 (#6179)
    • Added support for charset-normalizer 2.1.0 (#6169)

    New Contributors

    Full Changelog: https://github.com/psf/requests/blob/main/HISTORY.md#2281-2022-06-29

    v2.28.0

    2.28.0 (2022-06-09)

    Deprecations

    • ⚠️ Requests has officially dropped support for Python 2.7. ⚠️ (#6091)
    • Requests has officially dropped support for Python 3.6 (including pypy3). (#6091)

    Improvements

    • Wrap JSON parsing issues in Request's JSONDecodeError for payloads without an encoding to make json() API consistent. (#6097)
    • Parse header components consistently, raising an InvalidHeader error in all invalid cases. (#6154)
    • Added provisional 3.11 support with current beta build. (#6155)
    • Requests got a makeover and we decided to paint it black. (#6095)

    Bugfixes

    • Fixed bug where setting CURL_CA_BUNDLE to an empty string would disable cert verification. All Requests 2.x versions before 2.28.0 are affected. (#6074)
    • Fixed urllib3 exception leak, wrapping urllib3.exceptions.SSLError with requests.exceptions.SSLError for content and iter_content. (#6057)
    • Fixed issue where invalid Windows registry entires caused proxy resolution to raise an exception rather than ignoring the entry. (#6149)
    • Fixed issue where entire payload could be included in the error message for JSONDecodeError. (#6079)

    New Contributors

    ... (truncated)

    Changelog

    Sourced from requests's changelog.

    2.28.1 (2022-06-29)

    Improvements

    • Speed optimization in iter_content with transition to yield from. (#6170)

    Dependencies

    • Added support for chardet 5.0.0 (#6179)
    • Added support for charset-normalizer 2.1.0 (#6169)

    2.28.0 (2022-06-09)

    Deprecations

    • ⚠️ Requests has officially dropped support for Python 2.7. ⚠️ (#6091)
    • Requests has officially dropped support for Python 3.6 (including pypy3.6). (#6091)

    Improvements

    • Wrap JSON parsing issues in Request's JSONDecodeError for payloads without an encoding to make json() API consistent. (#6097)
    • Parse header components consistently, raising an InvalidHeader error in all invalid cases. (#6154)
    • Added provisional 3.11 support with current beta build. (#6155)
    • Requests got a makeover and we decided to paint it black. (#6095)

    Bugfixes

    • Fixed bug where setting CURL_CA_BUNDLE to an empty string would disable cert verification. All Requests 2.x versions before 2.28.0 are affected. (#6074)
    • Fixed urllib3 exception leak, wrapping urllib3.exceptions.SSLError with requests.exceptions.SSLError for content and iter_content. (#6057)
    • Fixed issue where invalid Windows registry entires caused proxy resolution to raise an exception rather than ignoring the entry. (#6149)
    • Fixed issue where entire payload could be included in the error message for JSONDecodeError. (#6036)

    2.27.1 (2022-01-05)

    Bugfixes

    • Fixed parsing issue that resulted in the auth component being dropped from proxy URLs. (#6028)

    2.27.0 (2022-01-03)

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 0
  • Bump validators from 0.18.2 to 0.20.0

    Bump validators from 0.18.2 to 0.20.0

    Bumps validators from 0.18.2 to 0.20.0.

    Changelog

    Sourced from validators's changelog.

    0.20.0 (2022-06-05) ^^^^^^^^^^^^^^^^^^^

    • Added ipv4 digit lenghts validation (#191, pull request courtesy of Norbiox)
    • Fixes error with international URLs that have more than 2 hyphens (#184, pull request courtesy of automationator)

    0.19.0 (2022-05-04) ^^^^^^^^^^^^^^^^^^^

    • Dropped py34 support
    • Improve IPv6 validation (#201, pull request courtesy of SimonIT)
    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 0
  • fds fails to pull dvc on windows

    fds fails to pull dvc on windows

    When running pull command on DagsHub remote. I receive dvc pull failure, so I have to manually pull dvc again.

    This issue permanent issue on windows.

    fds clone <remote> 
    
    

    It is not urgent issue, but in annoyance category.

    bug 
    opened by kingabzpro 3
Releases(v0.6.0)
  • v0.6.0(Nov 7, 2021)

  • v0.5.2(Oct 27, 2021)

  • v0.5.1(Oct 17, 2021)

    Changes

    🐛 Bug Fixes

    • Make hooks work as per commands @mohithg (#99)

    📦 Dependency updates

    • Bump humanize from 3.11.0 to 3.12.0 @dependabot (#103)
    Source code(tar.gz)
    Source code(zip)
  • v0.5.0(Sep 23, 2021)

    Changes

    🐛 Bug Fixes

    • Move __version__ to a version file @mohithg (#98)
    • Use dvc version 2.3.0 across the CLI @mohithg (#100)

    📦 Dependency updates

    • Use dvc version 2.3.0 across the CLI @mohithg (#100)
    Source code(tar.gz)
    Source code(zip)
  • v0.4.1(Sep 2, 2021)

    Changes

    🚀 Features

    • Make pip3 as default @mohithg (#96)
    • Fix dvc clone to ask if its git only project @mohithg (#88)

    🐛 Bug Fixes

    • Fix dvc clone to ask if its git only project @mohithg (#88)

    📦 Dependency updates

    • Update pytest requirement from ~=6.2.4 to ~=6.2.5 @dependabot (#95)
    Source code(tar.gz)
    Source code(zip)
  • v0.4.0(Aug 29, 2021)

    Changes

    🚀 Features

    • feat(run): implement pre-command hook runner to handle edge cases @pwoolvett (#91)

    🐛 Bug Fixes

    • Detect root of git and dvc @mohithg (#93)

    📝 Documentation updates

    • Fix documentation for fds clone @mohithg (#90)

    💥 Breaking changes

    • Detect root of git and dvc @mohithg (#93)
    Source code(tar.gz)
    Source code(zip)
  • v0.3.2(Aug 14, 2021)

  • v0.3.1v2(Aug 5, 2021)

    Changes

    🚀 Features

    • Integrate dvc auth with dvc pull @mohithg (#83)

    📦 Dependency updates

    • Bump progress from 1.5 to 1.6 @dependabot (#81)
    • Bump humanize from 3.10.0 to 3.11.0 @dependabot (#84)
    Source code(tar.gz)
    Source code(zip)
  • v0.3.0(Jul 29, 2021)

    Changes

    🚀 Features

    • Make fds push ask for credentials of user if its not authenticated already @mohithg (#80)

    🎨 UX improvements

    • Make fds push ask for credentials of user if its not authenticated already @mohithg (#80)
    Source code(tar.gz)
    Source code(zip)
  • v0.2.3(Jul 19, 2021)

  • v0.2.1(Jul 15, 2021)

    Changes

    🚀 Features

    • Add skip option to fds add wizard @mohithg (#74)

    🐛 Bug Fixes

    • Add ignored files to dvcignore too @mohithg (#75)

    📦 Dependency updates

    • Bump requests from 2.25.1 to 2.26.0 @dependabot (#77)
    • Bump humanize from 3.9.0 to 3.10.0 @dependabot (#72)

    🎨 UX improvements

    • Add skip option to fds add wizard @mohithg (#74)
    • Add ignored files to dvcignore too @mohithg (#75)
    Source code(tar.gz)
    Source code(zip)
  • v0.2.0(Jul 8, 2021)

    Changes

    🚀 Features

    • Feature to check for latest version before running a command @indweller (#73)
    • Implement clone command @mohithg (#53)

    🎨 UX improvements

    • Feature to check for latest version before running a command @indweller (#73)
    • Show an adding message before showing wizard in DVC add @mohithg (#70)
    Source code(tar.gz)
    Source code(zip)
  • v0.1.8(Jun 29, 2021)

    Changes

    📦 Dependency updates

    • Bump humanize from 3.8.0 to 3.9.0 @dependabot (#59)

    🚦 Tests

    • Fail tests on flake8 failure @mohithg (#63)

    🚀 Performance

    • Improve fds add performance by checking dvc-ignore when necessary @mohithg (#67)
    Source code(tar.gz)
    Source code(zip)
  • v0.1.7(Jun 17, 2021)

  • v0.1.6(Jun 14, 2021)

  • v0.1.5(Jun 14, 2021)

  • v0.1.4(Jun 7, 2021)

    Changes

    🐛 Bug Fixes

    • Add Integration tests and fix a bug with fds add <large file> that doesn't add into dvc @mohithg (#50)

    📦 Dependency updates

    • Bump humanize from 3.7.0 to 3.7.1 @dependabot (#51)
    • Bump humanize from 3.6.0 to 3.7.0 @dependabot (#49)
    • Bump pygit2 from 1.5.0 to 1.6.0 @dependabot (#48)

    🚦 Tests

    • Add Integration tests and fix a bug with fds add <large file> that doesn't add into dvc @mohithg (#50)
    Source code(tar.gz)
    Source code(zip)
  • v0.1.3(Jun 2, 2021)

    Changes

    📦 Dependency updates

    • Bump humanize from 3.5.0 to 3.6.0 @dependabot (#45)

    🎨 UX improvements

    • Improve UX by showing progress bar while adding large files to DVC and other minor fixes to improve usability @mohithg (#46)
    Source code(tar.gz)
    Source code(zip)
  • v0.1.2(May 27, 2021)

  • v0.1.1(May 27, 2021)

  • v0.1.0(May 26, 2021)

    Changes

    • Fix tests for fds @mohithg (#30)
    • Install DVC @mohithg (#8)
    • Implement commit command @mohithg (#7)
    • Dont track files/folders ignored in DVC and Git @mohithg (#6)
    • Fix feedback @mohithg (#5)
    • Remove dvc as a requirement for fds @mohithg (#4)
    • Add command @mohithg (#3)
    • Rename ff to fds @mohithg (#1)
    • Guy's feedback implementation @mohithg (#2)

    🚀 Features

    • Add -y option to let user decide to auto confirm all changed files @deanp70 (#26)
    • fds commands will now try to resume after installing dvc @guysmoilov (#22)

    🐛 Bug Fixes

    • Stop execution if command throws some error @mohithg (#28)
    • Removed non-working markdown from dvc install prompt @guysmoilov (#25)
    • Fix fds commit bug when there are actual changes @deanp70 (#12)

    🧰 Maintenance

    • Github action to upload to PyPI on GH release @guysmoilov (#33)
    • Github actions for python linting and testing @guysmoilov (#32)

    📝 Documentation updates

    • Update README.md @guysmoilov (#29)

    📦 Dependency updates

    • Update pytest requirement from ~=6.2.2 to ~=6.2.4 @dependabot (#23)
    • Bump humanize from 3.4.1 to 3.5.0 @dependabot (#24)

    🚦 Tests

    • Github actions for python linting and testing @guysmoilov (#32)
    Source code(tar.gz)
    Source code(zip)
Owner
DAGsHub
The home for data science collaboration
DAGsHub
Advanced Pandas Vault — Utilities, Functions and Snippets (by @firmai).

PandasVault ⁠— Advanced Pandas Functions and Code Snippets The only Pandas utility package you would ever need. It has no exotic external dependencies

Derek Snow 374 Jan 07, 2023
A DSL for data-driven computational pipelines

"Dataflow variables are spectacularly expressive in concurrent programming" Henri E. Bal , Jennifer G. Steiner , Andrew S. Tanenbaum Quick overview Ne

1.9k Jan 03, 2023
PyChemia, Python Framework for Materials Discovery and Design

PyChemia, Python Framework for Materials Discovery and Design PyChemia is an open-source Python Library for materials structural search. The purpose o

Materials Discovery Group 61 Oct 02, 2022
Anomaly Detection with R

AnomalyDetection R package AnomalyDetection is an open-source R package to detect anomalies which is robust, from a statistical standpoint, in the pre

Twitter 3.5k Dec 27, 2022
Produces a summary CSV report of an Amber Electric customer's energy consumption and cost data.

Amber Electric Usage Summary This is a command line tool that produces a summary CSV report of an Amber Electric customer's energy consumption and cos

Graham Lea 12 May 26, 2022
MIR Cheatsheet - Survival Guidebook for MIR Researchers in the Lab

MIR Cheatsheet - Survival Guidebook for MIR Researchers in the Lab

SeungHeonDoh 3 Jul 02, 2022
Using Data Science with Machine Learning techniques (ETL pipeline and ML pipeline) to classify received messages after disasters.

Using Data Science with Machine Learning techniques (ETL pipeline and ML pipeline) to classify received messages after disasters.

1 Feb 11, 2022
Python package for analyzing behavioral data for Brain Observatory: Visual Behavior

Allen Institute Visual Behavior Analysis package This repository contains code for analyzing behavioral data from the Allen Brain Observatory: Visual

Allen Institute 16 Nov 04, 2022
AptaMat is a simple script which aims to measure differences between DNA or RNA secondary structures.

AptaMAT Purpose AptaMat is a simple script which aims to measure differences between DNA or RNA secondary structures. The method is based on the compa

GEC UTC 3 Nov 03, 2022
Reading streams of Twitter data, save them to Kafka, then process with Kafka Stream API and Spark Streaming

Using Streaming Twitter Data with Kafka and Spark Reading streams of Twitter data, publishing them to Kafka topic, process message using Kafka Stream

Rustam Zokirov 1 Dec 06, 2021
A real-time financial data streaming pipeline and visualization platform using Apache Kafka, Cassandra, and Bokeh.

Realtime Financial Market Data Visualization and Analysis Introduction This repo shows my project about real-time stock data pipeline. All the code is

6 Sep 07, 2022
Python beta calculator that retrieves stock and market data and provides linear regressions.

Stock and Index Beta Calculator Python script that calculates the beta (β) of a stock against the chosen index. The script retrieves the data and resa

sammuhrai 4 Jul 29, 2022
.npy, .npz, .mtx converter.

npy-converter Matrix Data Converter. Expand matrix for multi-thread, multi-process Divid matrix for multi-thread, multi-process Support: .mtx, .npy, .

taka 1 Feb 07, 2022
This project is the implementation template for HW 0 and HW 1 for both the programming and non-programming tracks

This project is the implementation template for HW 0 and HW 1 for both the programming and non-programming tracks

Donald F. Ferguson 4 Mar 06, 2022
Titanic data analysis for python

Titanic-data-analysis This Repo is an analysis on Titanic_mod.csv This csv file contains some assumed data of the Titanic ship after sinking This full

Hardik Bhanot 1 Dec 26, 2021
A data parser for the internal syncing data format used by Fog of World.

A data parser for the internal syncing data format used by Fog of World. The parser is not designed to be a well-coded library with good performance, it is more like a demo for showing the data struc

Zed(Zijun) Chen 40 Dec 12, 2022
LynxKite: a complete graph data science platform for very large graphs and other datasets.

LynxKite is a complete graph data science platform for very large graphs and other datasets. It seamlessly combines the benefits of a friendly graphical interface and a powerful Python API.

124 Dec 14, 2022
The official pytorch implementation of ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Bias

ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Bias Introduction | Updates | Usage | Results&Pretrained Models | Statement | Intr

104 Nov 27, 2022
Open-source Laplacian Eigenmaps for dimensionality reduction of large data in python.

Fast Laplacian Eigenmaps in python Open-source Laplacian Eigenmaps for dimensionality reduction of large data in python. Comes with an wrapper for NMS

17 Jul 09, 2022