πŸ€• spelling exceptions builder for lazy people

Related tags

Text Data & NLPspellcheckdictionaryhacktoberfestpymorphy2
Overview

Yaspeller Dictionary (Auto)builder

CI

Usage

# this sample command generates `./yaspeller_report.json`
# yaspeller --report json --ignore-digits --ignore-text "'.*" --ignore-latin --only-errors --file-extensions ".md" --lang ru

python -m venv env
source env/bin/activate
pip install 
python src/dictionary.py yaspeller_report.json

Why

Yaspeller is nice, but there are too many anglicisms in a usual documentation. Normally you just want to ignore that, but there's the only possibility to add a regexp-array to ignore words.

This generates a array of dictionary words including all lexems for all cases like

[
    "[Π±Π‘]Π°Π³(Π°|Π°ΠΌ|Π°ΠΌΠΈ|Π°Ρ…|Π΅|ΠΈ|ΠΎΠ²|ΠΎΠΌ|Ρƒ)?",
    "[Π΄Π”]ΠΈΡ„Ρ„(Π°|Π°ΠΌ|Π°ΠΌΠΈ|Π°Ρ…|Π΅|ΠΎΠ²|ΠΎΠΌ|Ρƒ|Ρ‹)?",
    "[кК]ΠΎΠΌΠΌΠΈΡ‚(Π°|Π°ΠΌ|Π°ΠΌΠΈ|Π°Ρ…|Π΅|ΠΎΠ²|ΠΎΠΌ|Ρƒ|Ρ‹)?",
    "[пП]Π°Ρ‚Ρ‡ΠΈΠ½Π³(Π°|Π°ΠΌ|Π°ΠΌΠΈ|Π°Ρ…|Π΅|ΠΈ|ΠΎΠ²|ΠΎΠΌ|Ρƒ)?",
    "[Ρ€Π ]убист(Π°|Π°ΠΌ|Π°ΠΌΠΈ|Π°Ρ…|Π΅|ΠΎΠ²|ΠΎΠΌ|Ρƒ|Ρ‹)?",
    "[сБ]Π°ΠΌΠΎΠΎΡ€Π³Π°Π½ΠΈΠ·ΠΎΠ²Π°Π½Π½(ΠΎΠ³ΠΎ|ΠΎΠΌ|ΠΎΠΌΡƒ|ΡƒΡŽ|Ρ‹Π΅|Ρ‹ΠΉ|Ρ‹ΠΌ|Ρ‹ΠΌΠΈ|Ρ‹Ρ…)",
    "[Ρ‚Π’]ΠΈΠΊΠ΅Ρ‚(Π°|Π°ΠΌ|Π°ΠΌΠΈ|Π°Ρ…|Π΅|ΠΎΠ²|ΠΎΠΌ|Ρƒ|Ρ‹)?",
    "ΠΊΠΎΠΌΠΌΠΈΡ‚ΠΈΡ‚ΡŒ"
]

from yaspeller errors (in text format looking like)

Spelling check:
βœ— www.ruby-lang.org/ru/community/ruby-core/index.md 130 ms
-----
Typos: 9
1. ΠΏΠ°Ρ‚Ρ‡ΠΈΠ½Π³ΠΎΠΌ (36:27)
2. ΠΊΠΎΠΌΠΌΠΈΡ‚ΠΈΡ‚ΡŒ (68:32, suggest: ΠΊΠΎΠΌΠΈΡ‚Π΅Ρ‚)
3. Π±Π°Π³Π°Ρ… (75:15, suggest: Π±ΠΎΠ³Π°Ρ…, Π±Π°ΠΊΠ°Ρ…, Π±Π΅Π³Π°Ρ…)
4. Π±Π°Π³ΠΈ (89:24, suggest: Π±Π°Π³Π³ΠΈ)
5. Π±Π°Π³ (96:25)
6. Ρ‚ΠΈΠΊΠ΅Ρ‚ (107:14, suggest: этикСт)
7. Π΄ΠΈΡ„Ρ„ (115:18)
8. ΠΊΠΎΠΌΠΌΠΈΡ‚Ρƒ (147:24, suggest: ΠΊΠΎΠΌΠ΅Ρ‚Ρƒ, ΠΊΠΎΠΌΠ½Π°Ρ‚Ρƒ)
9. ΠΊΠΎΠΌΠΌΠΈΡ‚Π° (148:58, suggest: ΠΊΠΎΠΌΠ΅Ρ‚Π°)
-----

Live example

Initially created for www.ruby-lang.org translations spellchecking

Owner
Vlad Bokov
Vlad Bokov
GitHub Repository
A2T: Towards Improving Adversarial Training of NLP Models (EMNLP 2021 Findings)

A2T: Towards Improving Adversarial Training of NLP Models This is the source code for the EMNLP 2021 (Findings) paper "Towards Improving Adversarial T

QData 17 Oct 15, 2022
Natural Language Processing library built with AllenNLP 🌲🌱

Custom Natural Language Processing with big and small models 🌲🌱

Recognai 65 Sep 13, 2022
The swas programming language

The Swas programming language This is a language that was made for fun. Installation Step 0: Make sure you have python installed Step 1. Clone this re

Swas.py 19 Jul 18, 2022
NLP command-line assistant powered by OpenAI

NLP command-line assistant powered by OpenAI

Axel 16 Dec 09, 2022
This repository details the steps in creating a Part of Speech tagger using Trigram Hidden Markov Models and the Viterbi Algorithm without using external libraries.

POS-Tagger This repository details the creation of a Part-of-Speech tagger using Trigram Hidden Markov Models to predict word tags in a word sequence.

Raihan Ahmed 1 Dec 09, 2021
Full Spectrum Bioinformatics - a free online text designed to introduce key topics in Bioinformatics using the Python

Full Spectrum Bioinformatics is a free online text designed to introduce key topics in Bioinformatics using the Python programming language. The text is written in interactive Jupyter Notebooks, whic

Jesse Zaneveld 33 Dec 28, 2022
This code extends the neural style transfer image processing technique to video by generating smooth transitions between several reference style images

Neural Style Transfer Transition Video Processing By Brycen Westgarth and Tristan Jogminas Description This code extends the neural style transfer ima

Brycen Westgarth 110 Jan 07, 2023
Bpe algorithm can finetune tokenizer - Bpe algorithm can finetune tokenizer

"# bpe_algorithm_can_finetune_tokenizer" this is an implyment for https://github

张博 1 Feb 02, 2022
A Domain Specific Language (DSL) for building language patterns. These can be later compiled into spaCy patterns, pure regex, or any other format

RITA DSL This is a language, loosely based on language Apache UIMA RUTA, focused on writing manual language rules, which compiles into either spaCy co

Ε arΕ«nas Navickas 60 Sep 26, 2022
[ICLR'19] Trellis Networks for Sequence Modeling

TrellisNet for Sequence Modeling This repository contains the experiments done in paper Trellis Networks for Sequence Modeling by Shaojie Bai, J. Zico

CMU Locus Lab 460 Oct 13, 2022
ACL22 paper: Imputing Out-of-Vocabulary Embeddings with LOVE Makes Language Models Robust with Little Cost

Imputing Out-of-Vocabulary Embeddings with LOVE Makes Language Models Robust with Little Cost LOVE is accpeted by ACL22 main conference as a long pape

Lihu Chen 32 Jan 03, 2023
Client library to download and publish models and other files on the huggingface.co hub

huggingface_hub Client library to download and publish models and other files on the huggingface.co hub Do you have an open source ML library? We're l

Hugging Face 644 Jan 01, 2023
Visual Automata is a Python 3 library built as a wrapper for Caleb Evans' Automata library to add more visualization features.

Visual Automata Copyright 2021 Lewi Lie Uberg Released under the MIT license Visual Automata is a Python 3 library built as a wrapper for Caleb Evans'

Lewi Uberg 55 Nov 17, 2022
Graph4nlp is the library for the easy use of Graph Neural Networks for NLP

Graph4NLP Graph4NLP is an easy-to-use library for R&D at the intersection of Deep Learning on Graphs and Natural Language Processing (i.e., DLG4NLP).

Graph4AI 1.5k Dec 23, 2022
New Modeling The Background CodeBase

Modeling the Background for Incremental Learning in Semantic Segmentation This is the updated official PyTorch implementation of our work: "Modeling t

Fabio Cermelli 9 Dec 28, 2022
A library for Multilingual Unsupervised or Supervised word Embeddings

MUSE: Multilingual Unsupervised and Supervised Embeddings MUSE is a Python library for multilingual word embeddings, whose goal is to provide the comm

Facebook Research 3k Jan 06, 2023
An open-source NLP library: fast text cleaning and preprocessing.

An open-source NLP library: fast text cleaning and preprocessing

Iaroslav 21 Mar 18, 2022
Flaxformer: transformer architectures in JAX/Flax

Flaxformer: transformer architectures in JAX/Flax Flaxformer is a transformer library for primarily NLP and multimodal research at Google. It is used

Google 114 Dec 29, 2022
Transformers and related deep network architectures are summarized and implemented here.

Transformers: from NLP to CV This is a practical introduction to Transformers from Natural Language Processing (NLP) to Computer Vision (CV) Introduct

Ibrahim Sobh 138 Dec 27, 2022
Lumped-element impedance calculator and frequency-domain plotter.

fastZ: Lumped-Element Impedance Calculator fastZ is a small tool for calculating and visualizing electrical impedance in Python. Features include: Sup

Wesley Hileman 47 Nov 18, 2022