Maha is a text processing library specially developed to deal with Arabic text.

What does this pull request change?

Closes #27.

Status (please check what you already did):

• [x] added some tests for the functionality
• [ ] updated the documentation
• [x] tox passes

What does this pull request change?

Resolves #15.

Status (please check what you already did):

• [x] added some tests for the functionality
• [x] updated the documentation
• [x] tox passes

What does this pull request change?

This PR introduces a new datasets module that offers an interface for all upcoming datasets. A new dataset, names, is released along with the module. It comprises 44,161 unique names with descriptions and name origin included for most names.

Link to updated docs: https://maha--40.org.readthedocs.build/en/40/overview.html#datasets

Status (please check what you already did):

• [x] added some tests for the functionality
• [x] updated the documentation
• [x] tox passes

What does this pull request change?

Upgrades to new type annotations style.

Status (please check what you already did):

• [ ] added some tests for the functionality
• [ ] updated the documentation
• [x] tox passes

What does this pull request change?

• Removes datasets module.
• Datasets are now hosted here

Status (please check what you already did):

• [ ] added some tests for the functionality
• [ ] updated the documentation
• [x] tox passes

What does this pull request change?

Adds #24. Depends on #40

Status (please check what you already did):

• [x] added some tests for the functionality
• [x] updated the documentation
• [x] tox passes

What does this pull request change?

• Closes #23
• Adds 3 deprecation decorators; for functions, for parameters, for default parameters.

Status (please check what you already did):

• [x] added some tests for the functionality
• [ ] updated the documentation
• [x] tox passes

This is an auto-generated PR to prepare for the next release of Maha. The following changes were automatically made:

• Generated changelogs for release v0.3.0.
• Bumped pypi version to v0.3.0.
• Updated the citation information.

What does this pull request change?

Closes #48.

Status (please check what you already did):

• [x] added some tests for the functionality
• [ ] updated the documentation
• [x] tox passes

What does this pull request change?

Closes #25

Status (please check what you already did):

• [x] added some tests for the functionality
• [ ] updated the documentation
• [x] tox passes

This is an auto-generated PR to prepare for the next release of Maha. The following changes were automatically made:

• Generated changelogs for release v0.2.0.
• Bumped pypi version to v0.2.0.
• Updated the citation information.

Check the support for python 3,10

What does this pull request change? It checks if the library is supporting python 3.10.

• ...

Status (please check what you already did):

• [ ] added some tests for the functionality
• [ ] updated the documentation
• [ ] tox passes

updates:

• github.com/pre-commit/pre-commit-hooks: v4.3.0 → v4.4.0
• github.com/psf/black: 22.6.0 → 22.12.0
• github.com/pycqa/isort: 5.10.1 → 5.11.4
• github.com/asottile/pyupgrade: v2.37.3 → v3.3.1

What problem are you trying to solve?

Currently, the cleaner functions do not consider two strings similar if they have different Harakat/diacritics, which is the correct behavior. However, it would be great if the user had the option to ignore Harakat when comparing strings.

Examples (if relevant)

Current:

>> from maha.cleaners.functions import remove
>> output = remove("يُدَرِّسُ اللُّغَةَ العَرَبِيَّةَ الفُصْحَى", custom_expressions=r"اللغة")
>> output
يُدَرِّسُ اللُّغَةَ العَرَبِيَّةَ الفُصْحَى

Suggested:

>> from maha.cleaners.functions import remove
>> remove("يُدَرِّسُ اللُّغَةَ العَرَبِيَّةَ الفُصْحَى", custom_expressions=r"اللغة", ignore_harakat=True)
>> output
يُدَرِّسُ العَرَبِيَّةَ الفُصْحَى

Definition of Done

• It must adhere to the coding style used in the defined cleaner functions.
• The implementation should cover most use cases.
• Adding tests

What happened?

The name parser extracted wrong name likes : بي, شكرا.

Example: text: أريد البحث في سجل الإنفاق الخاص بي [Dimension(body=بي, value=بي, start=32, end=34, dimension_type=DimensionType.NAME)]

I expect to extract the names on the name dataset only.

Python version

3.8

What operating system are you using?

Linux

Code to reproduce the issue

>>> from maha.parsers.functions import parse_dimension
>>> text = `أريد البحث في سجل الإنفاق الخاص بي`
>>> extracted = parse_dimension(text, names=True)
[Dimension(body=بي, value=بي, start=32, end=34, dimension_type=DimensionType.NAME)]

Relevant log output

No response

What problem are you trying to solve?

Parsing the duration from the text that has the difference between the two dates.

Examples (if relevant)

>>> from maha.parsers.functions import parse_dimension
>>> output = parse_dimension('عن ربع نمو سكان العالم القديم والتحضر بين 1700 و 1900 ميلادي', duration=True)[0].value
>>> output
DurationValue(values=[ValueUnit(value=200, unit=<DurationUnit.YEARS: 7>)], normalized_unit=<DurationUnit.SECONDS: 1>)

Definition of Done

• It must adhere to the coding style used in the defined dimensions, duration dimension.
• The implementation should cover most use cases.
• Adding tests

What problem are you trying to solve?

Adding the parser functionality to Processors to parse different dimensions.

Examples (if relevant)

>>> from pathlib import Path
>>> import maha
>>> resource_path = Path(maha.__file__).parents[1] / "sample_data/tweets.txt"
>>> data = resource_path.read_text()
>>> print(data)

الساعة الآن 12:00 في اسبانيا 🇪🇸, انتهى بشكل رسمي عقد الأسطورة ليو ميسي مع برشلونة . .
طبعا بكونو حاطين المكيف ع٣ مئوية وخود تقلبات وبرد وحر وCNS وزعيق المراقب وألف نيلة وقر فتحت اشوف درجة الحرارة هتبقي كام يو الامتحان لقيتها ٤٢ والامتحان الساعه ١ فعايز انورماليز اننا ننزل بالفالنه الحمالات Hot fac
يسعدلي مساكم ❤🌹 شرح كلمة zwa هالمنشور رح تلاقو (zwar) سهل و لذيذ (aber) ناقصو شوية ملح وكزبر #منقو
مـعلش استحملوني ب الاصفر هالفتره 💛 #ريشـه هههههههه
لما حد يسالني بتختفي كتير لية =..
زيِّنوا ليلة الجمع بالصلاة على النَّبِيِّ ﷺ" ❤
#Windows11 is on the horizon. What feature are you looking forward to
Get vaccinate #savethesaviour
Today I am beginning project on 10 days duratio #30daysofcod #DEVCommunit

>>> from maha.processors import FileProcessor
>>> proc = FileProcessor(resource_path)
>>> parsed = proc.parse_dimension(time=True)
[Dimension(body=الساعة الآن 12:00, value=TimeValue(years=0, months=0, days=0, hours=0, minutes=0, seconds=0, hour=12, minute=0, second=0, microsecond=0), start=0, end=17, dimension_type=DimensionType.TIME),
 Dimension(body=الساعه ١, value=TimeValue(hour=1, minute=0, second=0, microsecond=0), start=238, end=246, dimension_type=DimensionType.TIME),
 Dimension(body=ليلة, value=TimeValue(am_pm='PM'), start=491, end=495, dimension_type=DimensionType.TIME)]

Definition of Done

• It must adhere to the coding style.
• The implementation should cover most use cases.
• Adding tests.