MMDA - multimodal document analysis
This is work in progress...
Setup
conda create -n mmda python=3.8
pip install -r requirements.txt
Parsers
-
SymbolScraper - Apache 2.0
- Quoted from their
README: From the main directory, issuemake. This will run the Maven build system, download dependencies, etc., compile source files and generate .jar files in./target. Finally, a bash scriptbin/sscraperis generated, so that the program can be easily used in different directories.
- Quoted from their
Library walkthrough
1. Creating a Document for the first time
In this example, we use the SymbolScraperParser. Each parser implements its own .parse().
import os
from mmda.parsers.symbol_scraper_parser import SymbolScraperParser
from mmda.types.document import Document
ssparser = SymbolScraperParser(sscraper_bin_path='...')
doc: Document = ssparser.parse(infile='...pdf', outdir='...', outfname='...json')
Because we provided outdir and outfname, the document is also serialized for you:
assert os.path.exists(os.path.join(outdir, outfname))
2. Loading a serialized Document
Each parser implements its own .load().
doc: Document = ssparser.load(infile=os.path.join(outdir, outfname))
3. Iterating through a Document
The minimum requirement for a Document is its .text field, which is just a
.
But the usefulness of this library really is when you have multiple different ways of segmenting the .text. For example:
for page in doc.pages:
print(f'\n=== PAGE: {page.id} ===\n\n')
for row in page.rows:
print(row.text)
shows two nice aspects of this library:
-
Documentprovides iterables for different segmentations oftext. Options includepages, tokens, rows, sents, blocks. Not every Parser will provide every segmentation, though. For example,SymbolScraperParseronly providespages, tokens, rows. -
Each one of these segments (precisely,
DocSpanobjects) is aware of (and can access) other segment types. For example, you can callpage.rowsto get all Rows that intersect a particular Page. Or you can callsent.tokensto get all Tokens that intersect a particular Sentence. Or you can callsent.blockto get the Block(s) that intersect a particular Sentence. These indexes are built dynamically when theDocumentis created and each time a newDocSpantype is loaded. In the extreme, one can do:
for page in doc.pages:
for block in page.blocks:
for sent in block.sents:
for row in sent.rows:
for token in sent.tokens:
pass
4. Loading new DocSpan type
Not all Documents will have all segmentations available at creation time. You may need to load new definitions to an existing Document.
It's strongly recommended to create the full Document using a Parser.load() but if you need to build it up step by step using the DocSpan class and Document.load() method:
from mmda.types.span import Span
from mmda.types.document import Document, DocSpan, Token, Page, Row, Sent, Block
doc: Document(text='I live in New York. I read the New York Times.')
page_jsons = [{'start': 0, 'end': 46, 'id': 0}]
sent_jsons = [{'start': 0, 'end': 19, 'id': 0}, {'start': 20, 'end': 46, 'id': 1}]
pages = [
DocSpan.from_span(span=Span.from_json(span_json=page_json),
doc=doc,
span_type=Page)
for page_json in page_jsons
]
sents = [
DocSpan.from_span(span=Span.from_json(span_json=sent_json),
doc=doc,
span_type=Sent)
for sent_json in sent_jsons
]
doc.load(sents=sents, pages=pages)
assert doc.sents
assert doc.pages
5. Changing the Document
We currently don't support any nice tools for mutating the data in a Document once it's been created, aside from loading new data. Do at your own risk.
But a note -- If you're editing something (e.g. replacing some DocSpan in tokens), always call:
Document._build_span_type_to_spans()
Document._build_span_type_to_index()
to keep the indices up-to-date with your modified DocSpan.
12 Nov 25, 2022
66 Dec 14, 2022
58 Dec 07, 2022
4 Nov 24, 2021
11.4k Jan 04, 2023
185 Dec 21, 2022
2 Jan 04, 2023
40 Nov 25, 2022
1 Jan 24, 2022
4 Feb 19, 2022
358 Dec 24, 2022
690 Jan 04, 2023
5.4k Jan 03, 2023
186 Dec 29, 2022
12 Oct 15, 2022
829 Jan 07, 2023
22 Dec 29, 2022
61 Sep 18, 2022
16 Oct 08, 2022
6.4k Dec 27, 2022