Python codecs extension featuring CLI tools for encoding/decoding anything

Related tags

Command-line Toolspythonencodingcryptographydecoderdnamorsebase58codecbraillebase62ctfencodingsbaseleetspeakctf-toolsbase36encoder-decodercodecsbase100base45
Overview

CodExt Tweet

Encode/decode anything.

PyPi Read The Docs Build Status Coverage Status Python Versions Requirements Status Known Vulnerabilities DOI License

This library extends the native codecs library (namely for adding new custom encodings and character mappings) and provides a myriad of new encodings (static or parametrized, like rot or xor), hence its named combining CODecs EXTension.

$ pip install codext
Want to contribute a new codec ? Want to contribute a new macro ?
Check the documentation first
Then PR your new codec		 PR your updated version of macros.json

πŸ” Demonstrations

Using CodExt from the command line

Using base tools from the command line

Using the debase command line tool

πŸ’» Usage (main CLI tool) Tweet on codext

$ codext -i test.txt encode dna-1
GTGAGCGGGTATGTGA

$ echo -en "test" | codext encode morse
- . ... -

$ echo -en "test" | codext encode braille
β žβ ‘β Žβ ž

$ echo -en "test" | codext encode base100
πŸ‘«πŸ‘œπŸ‘ͺπŸ‘«

Chaining codecs

$ echo -en "Test string" | codext encode reverse
gnirts tseT

$ echo -en "Test string" | codext encode reverse morse
--. -. .. .-. - ... / - ... . -

$ echo -en "Test string" | codext encode reverse morse dna-2
AGTCAGTCAGTGAGAAAGTCAGTGAGAAAGTGAGTGAGAAAGTGAGTCAGTGAGAAAGTCAGAAAGTGAGTGAGTGAGAAAGTTAGAAAGTCAGAAAGTGAGTGAGTGAGAAAGTGAGAAAGTC

$ echo -en "Test string" | codext encode reverse morse dna-2 octal
101107124103101107124103101107124107101107101101101107124103101107124107101107101101101107124107101107124107101107101101101107124107101107124103101107124107101107101101101107124103101107101101101107124107101107124107101107124107101107101101101107124124101107101101101107124103101107101101101107124107101107124107101107124107101107101101101107124107101107101101101107124103

$ echo -en "AGTCAGTCAGTGAGAAAGTCAGTGAGAAAGTGAGTGAGAAAGTGAGTCAGTGAGAAAGTCAGAAAGTGAGTGAGTGAGAAAGTTAGAAAGTCAGAAAGTGAGTGAGTGAGAAAGTGAGAAAGTC" | codext -d dna-2 morse reverse
test string

Using macros

$ codext add-macro my-encoding-chain gzip base63 lzma base64

$ codext list macros
example-macro, my-encoding-chain

$ echo -en "Test string" | codext encode my-encoding-chain
CQQFAF0AAIAAABuTgySPa7WaZC5Sunt6FS0ko71BdrYE8zHqg91qaqadZIR2LafUzpeYDBalvE///ug4AA==

$ codext remove-macro my-encoding-chain

$ codext list macros
example-macro

πŸ’» Usage (base CLI tool) Tweet on debase

$ echo "Test string !" | base122
*.7!ft9οΏ½-f9Γ‚

$ echo "Test string !" | base91 
"ONK;WDZM%Z%xE7L

$ echo "Test string !" | base91 | base85
B2P|BJ6A+nO(j|-cttl%

$ echo "Test string !" | base91 | base85 | base36 | base58-flickr
QVx5tvgjvCAkXaMSuKoQmCnjeCV1YyyR3WErUUErFf

$ echo "Test string !" | base91 | base85 | base36 | base58-flickr | base58-flickr -d | base36 -d | base85 -d | base91 -d
Test string !

$ echo "Test string !" | base91 | base85 | base36 | base58-flickr | debase -m 3
Test string !

$ echo "Test string !" | base91 | base85 | base36 | base58-flickr | debase -f Test
Test string !

πŸ’» Usage (Python)

Getting the list of available codecs:

>> codext.encode("this is a test", "base58-ripple") 'jo9rA2LQwr44eBmZK7E' >>> codext.encode("this is a test", "base58-url") 'JN91Wzkpa1nnDbLyjtf' >>> codecs.encode("this is a test", "base100") 'πŸ‘«πŸ‘ŸπŸ‘ πŸ‘ͺπŸ—πŸ‘ πŸ‘ͺπŸ—πŸ‘˜πŸ—πŸ‘«πŸ‘œπŸ‘ͺπŸ‘«' >>> codecs.decode("πŸ‘«πŸ‘ŸπŸ‘ πŸ‘ͺπŸ—πŸ‘ πŸ‘ͺπŸ—πŸ‘˜πŸ—πŸ‘«πŸ‘œπŸ‘ͺπŸ‘«", "base100") 'this is a test' >>> for i in range(8): print(codext.encode("this is a test", "dna-%d" % (i + 1))) GTGAGCCAGCCGGTATACAAGCCGGTATACAAGCAGACAAGTGAGCGGGTATGTGA CTCACGGACGGCCTATAGAACGGCCTATAGAACGACAGAACTCACGCCCTATCTCA ACAGATTGATTAACGCGTGGATTAACGCGTGGATGAGTGGACAGATAAACGCACAG AGACATTCATTAAGCGCTCCATTAAGCGCTCCATCACTCCAGACATAAAGCGAGAC TCTGTAAGTAATTCGCGAGGTAATTCGCGAGGTAGTGAGGTCTGTATTTCGCTCTG TGTCTAACTAATTGCGCACCTAATTGCGCACCTACTCACCTGTCTATTTGCGTGTC GAGTGCCTGCCGGATATCTTGCCGGATATCTTGCTGTCTTGAGTGCGGGATAGAGT CACTCGGTCGGCCATATGTTCGGCCATATGTTCGTCTGTTCACTCGCCCATACACT >>> codext.decode("GTGAGCCAGCCGGTATACAAGCCGGTATACAAGCAGACAAGTGAGCGGGTATGTGA", "dna-1") 'this is a test' >>> codecs.encode("this is a test", "morse") '- .... .. ... / .. ... / .- / - . ... -' >>> codecs.decode("- .... .. ... / .. ... / .- / - . ... -", "morse") 'this is a test' >>> with open("morse.txt", 'w', encoding="morse") as f: f.write("this is a test") 14 >>> with open("morse.txt",encoding="morse") as f: f.read() 'this is a test' >>> codext.decode(""" = X : x n r y Y y p a ` n | a o h ` g o z """, "whitespace-after+before") 'CSC{not_so_invisible}' >>> print(codext.encode("An example test string", "baudot-tape")) ***.** . * ***.* * . .* * .* . * ** .* ***.** ** .** .* * . * *. * .* * *. * *. * * . * *. * *. * ***. *.* ***.* * .* "> 
>>> import codext

>>> codext.list()
['ascii85', 'base85', 'base100', 'base122', ..., 'tomtom', 'dna', 'html', 'markdown', 'url', 'resistor', 'sms', 'whitespace', 'whitespace-after-before']

>>> codext.encode("this is a test", "base58-bitcoin")
'jo91waLQA1NNeBmZKUF'

>>> codext.encode("this is a test", "base58-ripple")
'jo9rA2LQwr44eBmZK7E'

>>> codext.encode("this is a test", "base58-url")
'JN91Wzkpa1nnDbLyjtf'

>>> codecs.encode("this is a test", "base100")
'πŸ‘«πŸ‘ŸπŸ‘ πŸ‘ͺπŸ—πŸ‘ πŸ‘ͺπŸ—πŸ‘˜πŸ—πŸ‘«πŸ‘œπŸ‘ͺπŸ‘«'

>>> codecs.decode("πŸ‘«πŸ‘ŸπŸ‘ πŸ‘ͺπŸ—πŸ‘ πŸ‘ͺπŸ—πŸ‘˜πŸ—πŸ‘«πŸ‘œπŸ‘ͺπŸ‘«", "base100")
'this is a test'

>>> for i in range(8):
        print(codext.encode("this is a test", "dna-%d" % (i + 1)))
GTGAGCCAGCCGGTATACAAGCCGGTATACAAGCAGACAAGTGAGCGGGTATGTGA
CTCACGGACGGCCTATAGAACGGCCTATAGAACGACAGAACTCACGCCCTATCTCA
ACAGATTGATTAACGCGTGGATTAACGCGTGGATGAGTGGACAGATAAACGCACAG
AGACATTCATTAAGCGCTCCATTAAGCGCTCCATCACTCCAGACATAAAGCGAGAC
TCTGTAAGTAATTCGCGAGGTAATTCGCGAGGTAGTGAGGTCTGTATTTCGCTCTG
TGTCTAACTAATTGCGCACCTAATTGCGCACCTACTCACCTGTCTATTTGCGTGTC
GAGTGCCTGCCGGATATCTTGCCGGATATCTTGCTGTCTTGAGTGCGGGATAGAGT
CACTCGGTCGGCCATATGTTCGGCCATATGTTCGTCTGTTCACTCGCCCATACACT
>>> codext.decode("GTGAGCCAGCCGGTATACAAGCCGGTATACAAGCAGACAAGTGAGCGGGTATGTGA", "dna-1")
'this is a test'

>>> codecs.encode("this is a test", "morse")
'- .... .. ... / .. ... / .- / - . ... -'

>>> codecs.decode("- .... .. ... / .. ... / .- / - . ... -", "morse")
'this is a test'

>>> with open("morse.txt", 'w', encoding="morse") as f:
	f.write("this is a test")
14

>>> with open("morse.txt",encoding="morse") as f:
	f.read()
'this is a test'

>>> codext.decode("""
      =            
              X         
   :            
      x         
  n  
    r 
        y   
      Y            
              y        
     p    
         a       
 `          
            n            
          |    
  a          
o    
       h        
          `            
          g               
           o 
   z      """, "whitespace-after+before")
'CSC{not_so_invisible}'

>>> print(codext.encode("An example test string", "baudot-tape"))
***.**
   . *
***.* 
*  .  
   .* 
*  .* 
   . *
** .* 
***.**
** .**
   .* 
*  .  
* *. *
   .* 
* *.  
* *. *
*  .  
* *.  
* *. *
***.  
  *.* 
***.* 
 * .*

πŸ“ƒ List of codecs

BaseXX

  • ascii85: classical ASCII85 (Python3 only)
  • baseN: see base encodings (incl base32, 36, 45, 58, 62, 63, 64, 91, 100, 122)
  • base-genericN: see base encodings ; supports any possible base

Binary

  • baudot: supports CCITT-1, CCITT-2, EU/FR, ITA1, ITA2, MTK-2 (Python3 only), UK, ...
  • baudot-spaced: variant of baudot ; groups of 5 bits are whitespace-separated
  • baudot-tape: variant of baudot ; outputs a string that looks like a perforated tape
  • bcd: Binary Coded Decimal, encodes characters from their (zero-left-padded) ordinals
  • bcd-extended0: variant of bcd ; encodes characters from their (zero-left-padded) ordinals using prefix bits 0000
  • bcd-extended1: variant of bcd ; encodes characters from their (zero-left-padded) ordinals using prefix bits 1111
  • excess3: uses Excess-3 (aka Stibitz code) binary encoding to convert characters from their ordinals
  • gray: aka reflected binary code
  • manchester: XORes each bit of the input with 01
  • manchester-inverted: variant of manchester ; XORes each bit of the input with 10
  • rotateN: rotates characters by the specified number of bits (N belongs to [1, 7] ; Python 3 only)

Common

  • a1z26: keeps words whitespace-separated and uses a custom character separator
  • cases: set of case-related encodings (including camel-, kebab-, lower-, pascal-, upper-, snake- and swap-case, slugify, capitalize, title)
  • dummy: set of simple encodings (including reverse and word-reverse)
  • octal: dummy octal conversion (converts to 3-digits groups)
  • octal-spaced: variant of octal ; dummy octal conversion, handling whitespace separators
  • ordinal: dummy character ordinals conversion (converts to 3-digits groups)
  • ordinal-spaced: variant of ordinal ; dummy character ordinals conversion, handling whitespace separators

Compression

  • gzip: standard Gzip compression/decompression
  • lz77: compresses the given data with the algorithm of Lempel and Ziv of 1977
  • lz78: compresses the given data with the algorithm of Lempel and Ziv of 1978
  • pkzip_deflate: standard Zip-deflate compression/decompression
  • pkzip_bzip2: standard BZip2 compression/decompression
  • pkzip_lzma: standard LZMA compression/decompression

Cryptography

  • affine: aka Affine Cipher
  • atbash: aka Atbash Cipher
  • bacon: aka Baconian Cipher
  • barbie-N: aka Barbie Typewriter (N belongs to [1, 4])
  • citrix: aka Citrix CTX1 passord encoding
  • rotN: aka Caesar cipher (N belongs to [1,25])
  • scytaleN: encrypts using the number of letters on the rod (N belongs to [1,[)
  • shiftN: shift ordinals (N belongs to [1,255])
  • xorN: XOR with a single byte (N belongs to [1,255])

Languages

  • braille: well-known braille language (Python 3 only)
  • ipsum: aka lorem ipsum
  • leetspeak: based on minimalistic elite speaking rules
  • morse: uses whitespace as a separator
  • navajo: only handles letters (not full words from the Navajo dictionary)
  • radio: aka NATO or radio phonetic alphabet
  • southpark: converts letters to Kenny's language from Southpark (whitespace is also handled)
  • southpark-icase: case insensitive variant of southpark
  • tomtom: similar to morse, using slashes and backslashes

Others

  • dna: implements the 8 rules of DNA sequences (N belongs to [1,8])
  • html: implements entities according to this reference
  • letter-indices: encodes consonants and/or vowels with their corresponding indices
  • markdown: unidirectional encoding from Markdown to HTML
  • url: aka URL encoding

Steganography

  • klopf: aka Klopf code ; Polybius square with trivial alphabetical distribution
  • resistor: aka resistor color codes
  • sms: also called T9 code ; uses "-" as a separator for encoding, "-" or "_" or whitespace for decoding
  • whitespace: replaces bits with whitespaces and tabs
  • whitespace_after_before: variant of whitespace ; encodes characters as new characters with whitespaces before and after according to an equation described in the codec name (e.g. "whitespace+2*after-3*before")

πŸ‘ Supporters

Stargazers repo roster for @dhondta/python-codext

Forkers repo roster for @dhondta/python-codext

Back to top

Comments
  • using Codext guess / Codext rank gives an error

    using Codext guess / Codext rank gives an error

    When I run it on linux and try to use "codext guess" or "codext rank" I get an error message saying:

    # echo "3yZe7d" | codext rank
Traceback (most recent call last):
  File "/usr/local/bin/codext", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/dist-packages/codext/__init__.py", line 184, in main
    args.include, args.exclude = __format_list(args.include), __format_list(args.exclude, False)
AttributeError: 'Namespace' object has no attribute 'include'
    opened by GitSunburn 1
  • UU decoding raises an exception in some cases

    UU decoding raises an exception in some cases 

    # cat raw.txt 
1Oh6axLwfecHErbVpRbtNj8t5JACsSQrofdnnMdQtBmoU8cQj6EyLcVRsQLz1MyWfXbqQDIc9wGyyBuH7uV95lBVpFGn3syGIw0IVLx8wJr4ABsIH9Q71hBH4AvIgljx7XnfjfmafahBNrPMDkK3dsJF0r41nzyMnOf7l36NcllAOgRLoB6Qh0APotZu6plYpkSiRCAkDKowOFm0mybKp336TAJe4JiDecD9hNbl5fBDLkGNYhmSkzOQzLBH1aPumW4o

# codext -i raw.txt decode base62
begin 666 <data>
M'XL( /H^)V("__-)SBB-<O7*+#;,K8PL\/ H\C#UR2LP= H)[email protected]\*"[email protected]
M\"T-\*@(#\]V\<A,*R^W\"@I,W9Q-,LU\8XR=$XM*TYR3PG,2G)+RO P#_'.
?"LRJS#)US0X(KPJH<[email protected](<TGR  #3A(K<90      
 
end

# codext -i raw.txt decode base62 | codext decode UU
Traceback (most recent call last):
  File "/usr/lib/python3.8/encodings/uu_codec.py", line 58, in uu_decode
    data = binascii.a2b_uu(s)
binascii.Error: Illegal char

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/jeane/.local/bin/codext", line 8, in <module>
    sys.exit(main())
  File "/home/jeane/.local/lib/python3.8/site-packages/codext/__init__.py", line 124, in main
    c = getattr(codecs, ["encode", "decode"][args.command == "decode"])(c, encoding, args.errors)
  File "/home/jeane/.local/lib/python3.8/site-packages/codext/__common__.py", line 691, in decode
    return lookup(encoding).decode(obj, errors)[0]
  File "/usr/lib/python3.8/encodings/uu_codec.py", line 62, in uu_decode
    data = binascii.a2b_uu(s[:nbytes])
binascii.Error: Illegal char
    opened by RomainJennes 1
  • Implement Rail Fence Cipher

    Implement Rail Fence Cipher

    Tests pass.

    RegEx might need a fix : rail-5-3- is recognized as a valid codec (ending dash is too much). Only rail-5-3 or rail-5-3-up should be recognized. Couldn't manage to get that right.

    When encoding, there might be trailing whitespaces (which is normal), but users might forget one when copy/pasting. I left them invisible to ease the integration with other tools.

    opened by smarbal 0
  • Add tap/knock language

    Add tap/knock language

    Tap encoding and decoding is implemented and added to the documentation. Couldn't make it work with add_map() due to problems with letter and word separations (single space/double space). Did my own encoding/decoding functions instead.

    opened by smarbal 0
  • [Snyk] Security upgrade markdown2 from 2.3.10 to 2.4.0

    [Snyk] Security upgrade markdown2 from 2.3.10 to 2.4.0

    Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • requirements.txt

    Vulnerabilities that will be fixed

    By pinning:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- high severity | 768/1000
    Why? Proof of Concept exploit, Recently disclosed, Has a fix available, CVSS 7.5 | Regular Expression Denial of Service (ReDoS)
    SNYK-PYTHON-MARKDOWN2-1063233 | markdown2:
    2.3.10 -> 2.4.0
    | No | Proof of Concept

    (*) Note that the real score may have changed since the PR was raised.

    Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the effected dependencies could be upgraded.

    Check the changes in this PR to ensure they won't cause issues with your project.

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    πŸ›  Adjust project settings

    πŸ“š Read more about Snyk's upgrade and patch logic

    opened by dhondta 0
  • pip 22.3 / python 3.10 warning on install

    pip 22.3 / python 3.10 warning on install

    Looks like there's a warning on the installation method via pip:

    pip install codext         
Collecting codext
  Downloading codext-1.14.0.tar.gz (116 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 116.2/116.2 kB 1.7 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Requirement already satisfied: six in ./venv/lib/python3.10/site-packages (from codext) (1.16.0)
Collecting markdown2>=2.4.0
  Downloading markdown2-2.4.6-py2.py3-none-any.whl (37 kB)
Installing collected packages: markdown2, codext
  DEPRECATION: codext is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559
  Running setup.py install for codext ... done
Successfully installed codext-1.14.0 markdown2-2.4.6
    opened by mikeatlas 0
Releases(1.10.1)
Owner
Alex
I'm a security professional, passionate about programming (especially Python) and cyber security.
Alex
GitHub Repository
Present - A terminal-based presentation tool with colors and effects.

present A terminal-based presentation tool with colors and effects. You can also play a codio (pre-recorded code block) on a slide. present is built o

Vinayak Mehta 4.2k Jan 03, 2023
A Python-based command prompt concept which includes windows command emulation.

PythonCMD A Python-based command prompt concept which includes windows command emulation. Current features: echo: Input your message and it will be cl

1 Feb 05, 2022
Unpacks things.

$ unp_ unp is a command line tool that can unpack archives easily. It mainly acts as a wrapper around other shell tools that you can find on v

Armin Ronacher 405 Jan 03, 2023
TermPair lets developers securely share and control terminals in real timeπŸ”’

View and control terminals from your browser with end-to-end encryption πŸ”’

Chad Smith 1.5k Jan 05, 2023
Format click help output nicely with rich.

rich-click Format click help output nicely with Rich. Click is a "Python package for creating beautiful command line interfaces". Rich is a "Python li

Phil Ewels 333 Jan 02, 2023
The command line interface for Gradient - Gradient is an an end-to-end MLOps platform

Gradient CLI Get started: Create Account β€’ Install CLI β€’ Tutorials β€’ Docs Resources: Website β€’ Blog β€’ Support β€’ Contact Sales Gradient is an an end-to

Paperspace 58 Dec 06, 2022
Wordle helper: help you print posible 5-character words based on you input

Wordle Helper This program help you print posible 5-character words based on you

Gwan Thanakrit Juthamongkhon 4 Jan 19, 2022
Pymongo based CLI client, to run operation on existing databases and collections

Mongodb-Operations-Console Pymongo based CLI client, to run operation on existing databases and collections Program developed by Gustavo Wydler Azuaga

Gus 1 Dec 01, 2021
ddgr is a cmdline utility to search DuckDuckGo (html version) from the terminal

ddgr is a cmdline utility to search DuckDuckGo (html version) from the terminal. While googler is extremely popular among cmdline users, in many forums the need of a similar utility for privacy-aware

PiΓ±a Colada 2.5k Dec 25, 2022
Apple Silicon 'top' CLI

asitop pip install asitop What A nvtop/htop style/inspired command line tool for Apple Silicon (aka M1) Macs. Note that it requires sudo to run due to

Timothy Liu 1.2k Dec 31, 2022
Faza - Faza terminal, Faza help to beginners for pen testing

Faza terminal simple tool for pen testers Use small letter only for commands Don't use space after command 'help' for more information Installation gi

Ag3ntQ 5 Feb 20, 2022
Joji convert a text to corresponding emoji if emoji is available

Joji Joji convert a text to corresponding emoji if emoji is available How it Works ? 1. There is a json file with emoji names as keys and correspondin

Gopikrishnan Sasikumar 28 Nov 26, 2022
A command-line tool to flash python code to Codey Rocky without having to use the online mblock5 IDE.

What? A command-line tool to flash python code to Codey Rocky without having to use the online mblock5 IDE. Description This is a very low-effort proj

1 Dec 29, 2021
Shortcut-Maker - It is a tool that can be set to run any tool with a single command

Shortcut-Maker It is a tool that can be set to run any tool with a single command Coded by Dave Smith(Owner of Sl Cyber Warriors) Command list πŸ‘‡ pkg

Dave Smith 10 Sep 14, 2022
This a simple tool to query the awesome ippsec.rocks website from your terminal

ippsec-cli This a simple tool to query the awesome ippsec.rocks website from your terminal Installation and usage cd /opt git clone https://github.com

stark0de 5 Nov 26, 2022
a GUI app base on warp-cli for linux

warp cloudflare gui a GUI app base on warp-cli for linux Installation read warp-cli install doc. install warp-cli and register with $ warp-cli registe

Moein Aghamirzaei 58 Jan 01, 2023
cmdpxl: a totally practical command-line image editor

cmdpxl: a totally practical command-line image editor

Jieruei Chang 476 Jan 07, 2023
A simple command line virtual operating system, written in python

Virtual operating system A simple virtual operating system written in python. (Under development). Currently, the following commands are supported: Co

B.Jothin kumar 7 Nov 15, 2022
Access hacksec.in from your command-line

Access hacksec.in from your command-line

hacksec.in 3 Oct 26, 2022
A 3D engine powered by ASCII art

3D engine powered by ASCII art

Lingdong Huang 48 Nov 16, 2022