Before You Start
Apple-peeler was written using python 3.9 (but it should be trivial to support earlier versions of python 3.5+).
Installation
pip install apple-peeler
Dependencies
BeautifulSoup 4, lxml, and click
Usage
Apple likes to move around the dictionaries location from macOS version to macOS version. So if the dictionaries are no longer at the path below you can tell apple-peeler where to look by exporting DICT_BASE in your environment or using the --base option below.
export DICT_BASE="/System/Library/AssetsV2/com_apple_MobileAsset_DictionaryServices_dictionaryOSX/"
After that, useage is straightforward.
Usage: apple-peeler [OPTIONS]
Extract XML from Apple Dictionary files.
Options:
--base DIRECTORY The root directory of the OS X dictionaries.
(Default: /System/Library/AssetsV2/com_apple
_MobileAsset_DictionaryServices_dictionaryOS
X/) [Env var DICT_BASE]
--out DIRECTORY The path to place extracted XML files.
-d, --dictionary [
all|Arabic - English|Danish|Duden Dictionary Data Set I|Dutch|
Dutch - English|French|French - English|French - German|German - English|
Hebrew|Hindi|Hindi - English|Indonesian - English|Italian|
Italian - English|Korean|Korean - English|New Oxford American Dictionary|
Norwegian|Oxford American Writer's Thesaurus|
Oxford Dictionary of English|Oxford Thesaurus of English|
Polish - English|Portuguese|Portuguese - English|Russian|
Russian - English|Sanseido Super Daijirin|
Sanseido The WISDOM English-Japanese Japanese-English Dictionary|
Simplified Chinese - English|Simplified Chinese - Japanese|Spanish|
Spanish - English|Swedish|Thai|Thai - English|
The Standard Dictionary of Contemporary Chinese|Traditional Chinese|
Traditional Chinese - English|Turkish|Vietnamese - English]
The dictionary to extract or 'all'.
(Default: all) [Accepts multiple]
--format-xml / --no-format-xml Format the XML files using BeautifulSoup.
(Default: False)
--debug Output debug information to STDERR.
(Default: False)
--help Show this message and exit.
Introduction
I need a ton of dictionary data for prototyping my learning a language tool, Parsnip, and licensing 40 dictionaries seems too expensive for a bootstrapper prototyping / working on an MVP (I look forward to the day this is no longer true). [Note: I am not planning to redistribute or otherwise use the data in an unlicensed manner.]
Parsnip uses Natural Language Processing and Dictionaries to decouple the word <-> sentence tug-of-war that's existed as long as flashcards have been used for language learning. I.e., should I make a word (concept) or a sentence (example) flashcard?
I care about what words I know for tracking purposes, but I want those words in context when I'm practicing. So the learning system breaks down sentences into lemmas (or dictionary form of a word) and a database of example sentences that the words appear in. This resolves the conceptual tug-of-war for flashcards.
But by removing reference data from the flashcards themselves, I need to integrate reference material directly into Parsnip's UI. JMDict is a great open source project for this, but that only covers a single language. So, I've been keeping my eyes open for people working on extracting the data from Apple's bundled dictionaries.
This has been a community effort that's spanned several years. My contribution is to collect the results, clear up some details about the file format, and package it into a general command-line tool.
References
This is inspired by Reverse-Engineering Apple Dictionary. And the discussion on Hacker News Hacker News: Reverse-Engineering Apple Dictionary (2020). Special thanks to tim-- and enragedcacti who introduced me to binwalk. And dunham who mentioned the random bytes looking like ints of payload sizes.
Additionally, I've found these posts informative:
- https://developer.apple.com/library/archive/documentation/UserExperience/Conceptual/DictionaryServicesProgGuide/prepare/prepare.html#//apple_ref/doc/uid/TP40006152-CH3-SW7
- https://jadedtuna.github.io/apple-dictionary/
- https://josephg.com/blog/reverse-engineering-apple-dictionaries/
- https://josephg.com/blog/apple-dictionaries-part-2/
- https://gist.github.com/josephg/5e134adf70760ee7e49d
7 Nov 21, 2022
859 Dec 29, 2022
5.8k Jan 3, 2023
163 Dec 31, 2022
2 Oct 31, 2022
2 Feb 11, 2022
514 Dec 31, 2022
567 Nov 30, 2022
5k Jan 4, 2023
1.7k Jan 5, 2023
2.3k Jan 2, 2023
33 Dec 8, 2022
27 Jun 20, 2022
46 Nov 14, 2022
15 May 17, 2022
![A repository that shares tuning results of trained models generated by TensorFlow / Keras. Post-training quantization (Weight Quantization, Integer Quantization, Full Integer Quantization, Float16 Quantization), Quantization-aware training. TensorFlow Lite. OpenVINO. CoreML. TensorFlow.js. TF-TRT. MediaPipe. ONNX. [.tflite,.h5,.pb,saved_model,tfjs,tftrt,mlmodel,.xml/.bin, .onnx]](https://user-images.githubusercontent.com/33194443/104581604-2592cb00-56a2-11eb-9610-5eaa0afb6e1f.png)
2.4k Jan 5, 2023
0 Jan 28, 2022
6 Aug 22, 2022
1 Dec 13, 2021
18 Dec 23, 2022
2 Jan 05, 2022
8 Feb 23, 2022
7 Nov 20, 2021
14 Aug 02, 2022
12 Nov 16, 2022
1 Nov 30, 2021
19 Sep 13, 2022
2 Mar 03, 2022
19 Jan 01, 2023
85 Jun 29, 2022
1.3k Dec 30, 2022
6 Jul 20, 2021
3 Oct 08, 2021
158 Dec 22, 2022
6 Aug 10, 2022
140 Dec 28, 2022
4 Nov 02, 2021
4 Dec 01, 2022