πŸ€• spelling exceptions builder for lazy people

Related tags

Text Data & NLPspellcheckdictionaryhacktoberfestpymorphy2
Overview

Yaspeller Dictionary (Auto)builder

CI

Usage

# this sample command generates `./yaspeller_report.json`
# yaspeller --report json --ignore-digits --ignore-text "'.*" --ignore-latin --only-errors --file-extensions ".md" --lang ru

python -m venv env
source env/bin/activate
pip install 
python src/dictionary.py yaspeller_report.json

Why

Yaspeller is nice, but there are too many anglicisms in a usual documentation. Normally you just want to ignore that, but there's the only possibility to add a regexp-array to ignore words.

This generates a array of dictionary words including all lexems for all cases like

[
    "[Π±Π‘]Π°Π³(Π°|Π°ΠΌ|Π°ΠΌΠΈ|Π°Ρ…|Π΅|ΠΈ|ΠΎΠ²|ΠΎΠΌ|Ρƒ)?",
    "[Π΄Π”]ΠΈΡ„Ρ„(Π°|Π°ΠΌ|Π°ΠΌΠΈ|Π°Ρ…|Π΅|ΠΎΠ²|ΠΎΠΌ|Ρƒ|Ρ‹)?",
    "[кК]ΠΎΠΌΠΌΠΈΡ‚(Π°|Π°ΠΌ|Π°ΠΌΠΈ|Π°Ρ…|Π΅|ΠΎΠ²|ΠΎΠΌ|Ρƒ|Ρ‹)?",
    "[пП]Π°Ρ‚Ρ‡ΠΈΠ½Π³(Π°|Π°ΠΌ|Π°ΠΌΠΈ|Π°Ρ…|Π΅|ΠΈ|ΠΎΠ²|ΠΎΠΌ|Ρƒ)?",
    "[Ρ€Π ]убист(Π°|Π°ΠΌ|Π°ΠΌΠΈ|Π°Ρ…|Π΅|ΠΎΠ²|ΠΎΠΌ|Ρƒ|Ρ‹)?",
    "[сБ]Π°ΠΌΠΎΠΎΡ€Π³Π°Π½ΠΈΠ·ΠΎΠ²Π°Π½Π½(ΠΎΠ³ΠΎ|ΠΎΠΌ|ΠΎΠΌΡƒ|ΡƒΡŽ|Ρ‹Π΅|Ρ‹ΠΉ|Ρ‹ΠΌ|Ρ‹ΠΌΠΈ|Ρ‹Ρ…)",
    "[Ρ‚Π’]ΠΈΠΊΠ΅Ρ‚(Π°|Π°ΠΌ|Π°ΠΌΠΈ|Π°Ρ…|Π΅|ΠΎΠ²|ΠΎΠΌ|Ρƒ|Ρ‹)?",
    "ΠΊΠΎΠΌΠΌΠΈΡ‚ΠΈΡ‚ΡŒ"
]

from yaspeller errors (in text format looking like)

Spelling check:
βœ— www.ruby-lang.org/ru/community/ruby-core/index.md 130 ms
-----
Typos: 9
1. ΠΏΠ°Ρ‚Ρ‡ΠΈΠ½Π³ΠΎΠΌ (36:27)
2. ΠΊΠΎΠΌΠΌΠΈΡ‚ΠΈΡ‚ΡŒ (68:32, suggest: ΠΊΠΎΠΌΠΈΡ‚Π΅Ρ‚)
3. Π±Π°Π³Π°Ρ… (75:15, suggest: Π±ΠΎΠ³Π°Ρ…, Π±Π°ΠΊΠ°Ρ…, Π±Π΅Π³Π°Ρ…)
4. Π±Π°Π³ΠΈ (89:24, suggest: Π±Π°Π³Π³ΠΈ)
5. Π±Π°Π³ (96:25)
6. Ρ‚ΠΈΠΊΠ΅Ρ‚ (107:14, suggest: этикСт)
7. Π΄ΠΈΡ„Ρ„ (115:18)
8. ΠΊΠΎΠΌΠΌΠΈΡ‚Ρƒ (147:24, suggest: ΠΊΠΎΠΌΠ΅Ρ‚Ρƒ, ΠΊΠΎΠΌΠ½Π°Ρ‚Ρƒ)
9. ΠΊΠΎΠΌΠΌΠΈΡ‚Π° (148:58, suggest: ΠΊΠΎΠΌΠ΅Ρ‚Π°)
-----

Live example

Initially created for www.ruby-lang.org translations spellchecking

Owner
Vlad Bokov
Vlad Bokov
GitHub Repository
Switch spaces for knowledge graph embeddings

SwisE Switch spaces for knowledge graph embeddings. Requirements: python3 pytorch numpy tqdm Reproduce the results To reproduce the reported results,

Shuai Zhang 4 Dec 01, 2021
Rootski - Full codebase for rootski.io (without the data)

πŸ“£ Welcome to the Rootski codebase! This is the codebase for the application run

Eric 20 Nov 18, 2022
Host your own GPT-3 Discord bot

GPT3 Discord Bot Host your own GPT-3 Discord bot i'd host and make the bot invitable myself, however GPT3 terms of service prohibit public use of GPT3

[something hillarious here] 8 Jan 07, 2023
Contract Understanding Atticus Dataset

Contract Understanding Atticus Dataset This repository contains code for the Contract Understanding Atticus Dataset (CUAD), a dataset for legal contra

The Atticus Project 273 Dec 17, 2022
sangha, pronounced "suhng-guh", is a social networking, booking platform where students and teachers can share their practice.

Flask React Project This is the backend for the Flask React project. Getting started Clone this repository (only this branch) git clone https://github

Courtney Newcomer 17 Sep 29, 2021
Flaxformer: transformer architectures in JAX/Flax

Flaxformer: transformer architectures in JAX/Flax Flaxformer is a transformer library for primarily NLP and multimodal research at Google. It is used

Google 114 Dec 29, 2022
AI_Assistant - This is a Python based Voice Assistant.

This is a Python based Voice Assistant. This was programmed to increase my understanding of python and also how the in-general Voice Assistants work.

1 Jan 06, 2022
Simple, Pythonic, text processing--Sentiment analysis, part-of-speech tagging, noun phrase extraction, translation, and more.

TextBlob: Simplified Text Processing Homepage: https://textblob.readthedocs.io/ TextBlob is a Python (2 and 3) library for processing textual data. It

Steven Loria 8.4k Dec 26, 2022
A Neural Language Style Transfer framework to transfer natural language text smoothly between fine-grained language styles like formal/casual, active/passive, and many more. Created by Prithiviraj Damodaran. Open to pull requests and other forms of collaboration.

Styleformer A Neural Language Style Transfer framework to transfer natural language text smoothly between fine-grained language styles like formal/cas

Prithivida 431 Dec 19, 2022
AIDynamicTextReader - A simple dynamic text reader based on Artificial intelligence

AI Dynamic Text Reader: This is a simple dynamic text reader based on Artificial

Md. Rakibul Islam 1 Jan 18, 2022
Beyond Masking: Demystifying Token-Based Pre-Training for Vision Transformers

beyond masking Beyond Masking: Demystifying Token-Based Pre-Training for Vision Transformers The code is coming Figure 1: Pipeline of token-based pre-

Yunjie Tian 23 Sep 27, 2022
TaCL: Improve BERT Pre-training with Token-aware Contrastive Learning

TaCL: Improve BERT Pre-training with Token-aware Contrastive Learning

Yixuan Su 26 Oct 17, 2022
Search-Engine - πŸ“– AI based search engine

Search Engine AI based search engine that was trained on 25000 samples, feel free to train on up to 1.2M sample from kaggle dataset, link below StackS

Vladislav Kruglikov 2 Nov 29, 2022
ConvBERT: Improving BERT with Span-based Dynamic Convolution

ConvBERT Introduction In this repo, we introduce a new architecture ConvBERT for pre-training based language model. The code is tested on a V100 GPU.

YITUTech 237 Dec 10, 2022
This is the 25 + 1 year anniversary version of the 1995 Rachford-Rice contest

Rachford-Rice Contest This is the 25 + 1 year anniversary version of the 1995 Rachford-Rice contest. Can you solve the Rachford-Rice problem for all t

13 Sep 20, 2022
πŸ¦… Pretrained BigBird Model for Korean (up to 4096 tokens)

Pretrained BigBird Model for Korean What is BigBird β€’ How to Use β€’ Pretraining β€’ Evaluation Result β€’ Docs β€’ Citation ν•œκ΅­μ–΄ | English What is BigBird? Bi

Jangwon Park 183 Dec 14, 2022
A retro text-to-speech bot for Discord

hawking A retro text-to-speech bot for Discord, designed to work with all of the stuff you might've seen in Moonbase Alpha, using the existing command

Nick Schorr 23 Dec 25, 2022
Implementation of ProteinBERT in Pytorch

ProteinBERT - Pytorch (wip) Implementation of ProteinBERT in Pytorch. Original Repository Install $ pip install protein-bert-pytorch Usage import torc

Phil Wang 92 Dec 25, 2022
Harvis is designed to automate your C2 Infrastructure.

Harvis Harvis is designed to automate your C2 Infrastructure, currently using Mythic C2. πŸ“Œ What is it? Harvis is a python tool to help you create mul

Thiago Mayllart 99 Oct 06, 2022
BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documents

BROS (BERT Relying On Spatiality) is a pre-trained language model focusing on text and layout for better key information extraction from documents. Given the OCR results of the document image, which

Clova AI Research 94 Dec 30, 2022