A python module for retrieving and parsing WHOIS data

Overview

pythonwhois

A WHOIS retrieval and parsing library for Python.

Dependencies

None! All you need is the Python standard library.

Instructions

The manual (including install instructions) can be found in the doc/ directory. A HTML version is also viewable here.

Goals

  • 100% coverage of WHOIS formats.
  • Accurate and complete data.
  • Consistently functional parsing; constant tests to ensure the parser isn't accidentally broken.

Features

  • WHOIS data retrieval
    • Able to follow WHOIS server redirects
    • Won't get stuck on multiple-result responses from verisign-grs
  • WHOIS data parsing
    • Base information (registrar, etc.)
    • Dates/times (registration, expiry, ...)
    • Full registrant information (!)
    • Nameservers
  • Optional WHOIS data normalization
    • Attempts to intelligently reformat WHOIS data for better (human) readability
    • Converts various abbreviation types to full locality names
      • Airport codes
      • Country names (2- and 3-letter ISO codes)
      • US states and territories
      • Canadian states and territories
      • Australian states
  • pwhois, a simple WHOIS tool using pythonwhois
    • Easily readable output format
    • Can also output raw WHOIS data
    • ... and JSON.
  • Automated testing suite
    • Will detect and warn about any changes in parsed data compared to previous runs
    • Guarantees that previously working WHOIS parsing doesn't unintentionally break when changing code

IP range WHOIS

pythonwhois does not yet support WHOIS lookups on IP ranges (including single IPs), although this will be added at some point in the future. In the meantime, consider using ipwhois - it offers functionality and an API similar to pythonwhois, but for IPs. It also supports delegated RWhois.

Do note that ipwhois does not offer a normalization feature, and does not (yet) come with a command-line tool. Additionally, ipwhois is maintained by Philip Hane and not by me; please make sure to file bugs relating to it in the ipwhois repository, not in that of pythonwhois.

Important update notes

2.4.0 and up: A lot of changes were made to the normalization, and the performance under Python 2.x was significantly improved. The average parsing time under Python 2.7 has dropped by 94% (!), and on my system averages out at 18ms. Performance under Python 3.x is unchanged. pythonwhois will now expand a lot of abbreviations in normalized mode, such as airport codes, ISO country codes, and US/CA/AU state abbreviations. The consequence of this is that the library is now bigger (as it ships a list of these abbreviations). Also note that there may be licensing consequences, in particular regarding the airport code database. More information about that can be found below.

2.3.0 and up: Python 3 support was fixed. Creation date parsing for contacts was fixed; correct timestamps will now be returned, rather than unformatted ones - if your application relies on the broken variant, you'll need to change your code. Some additional parameters were added to the net and parse methods to facilitate NIC handle lookups; the defaults are backwards-compatible, and these changes should not have any consequences for your code. Thai WHOIS parsing was implemented, but is a little spotty - data may occasionally be incorrectly split up. Please submit a bug report if you run across any issues.

2.2.0 and up: The internal workings of get_whois_raw have been changed, to better facilitate parsing of WHOIS data from registries that may return multiple partial matches for a query, such as whois.verisign-grs.com. This change means that, by default, get_whois_raw will now strip out the part of such a response that does not pertain directly to the requested domain. If your application requires an unmodified raw WHOIS response and is calling get_whois_raw directly, you should use the new never_cut parameter to keep pythonwhois from doing this post-processing. As this is a potentially breaking behaviour change, the minor version has been bumped.

It doesn't work!

  • It doesn't work at all?
  • It doesn't parse the data for a particular domain?
  • There's an inaccuracy in parsing the data for a domain, even just a small one?

If any of those apply, don't hesitate to file an issue! The goal is 100% coverage, and we need your feedback to reach that goal.

License

This library may be used under the WTFPL - or, if you take issue with that, consider it to be under the CC0.

Data sources

This library uses a number of third-party datasets for normalization:

Be aware that the OpenFlights database in particular has potential licensing consequences; if you do not wish to be bound by these potential consequences, you may simply delete the airports.dat file from your distribution. pythonwhois will assume there is no database available, and will not perform airport code conversion (but still function correctly otherwise). This also applies to other included datasets.

Contributing

Feel free to fork and submit pull requests (to the develop branch)! If you change any parsing or normalization logic, ensure to run the full test suite before opening a pull request. Instructions for that are below.

Please note that this project uses tabs for indentation.

All commands are relative to the root directory of the repository.

Pull requests that do not include output from test.py will be rejected!

Adding new WHOIS data to the testing set

pwhois --raw thedomain.com > test/data/thedomain.com

Checking the currently parsed data (while editing the parser)

./pwhois -f test/data/thedomain.com/ .

(don't forget the dot at the end!)

Marking the current parsed data as correct for a domain

Make sure to verify (using pwhois or otherwise) that the WHOIS data for the domain is being parsed correctly, before marking it as correct!

./test.py update thedomain.com

Running all tests

./test.py run all

Testing a specific domain

./test.py run thedomain.com

Running the full test suite including support for multiple python versions

tox

Generating documentation

You need ZippyDoc (which can be installed through pip install zippydoc).

zpy2html doc/*.zpy
Owner
Sven Slootweg
Sven Slootweg
Seamless deployment and management of cybersecurity solutions 🏗️

Description 🖼️ Background 👴🏼 Vision 📜 Concepts 💬 Solutions' Lifecycle. Operations ⭕ Functionalities 🚀 Supported Cybersecurity Solutions 📦 Insta

MutableSecurity 36 Nov 10, 2022
A Burp Pro extension that adds log4shell checks to Burp Scanner

scan4log4shell A Burp Pro extension that adds log4shell checks to Burp Scanner, written by Daniel Crowley of IBM X-Force Red. Installation To install

X-Force Red 26 Mar 15, 2022
This is a simple tool to create ZIP payloads using a provided wordlist for the symlink attack (present in some file upload vulnerabilities)

zip-symlink-payload-creator This is a simple tool to create ZIP payloads using a provided wordlist for the symlink attack (present in some file upload

stark0de 6 Aug 18, 2022
Wonk is a tool for combining a set of AWS policy files into smaller compiled policy sets.

Wonk is a tool for combining a set of AWS policy files into smaller compiled policy sets.

Amino, Inc 140 Dec 16, 2022
A hack for writing switch statements with type annotations in Python.

py_annotation_switch A hack for writing switch statements in type annotations for Python. Why should I use this? You most definitely should not use th

6 Oct 17, 2021
Volunteer & Campaign Management System

Cleansweep Requirements A Linux (or Mac OS X) node with the following software installed. Ubuntu 14.04 is preferred. PostgreSQL 9.3 database server Py

Aam Aadmi Party 39 May 24, 2022
A python base script from which you can hack or clone any person's facebook friendlist or followers accounts which have simple password

Hcoder This is a python base script from which you can hack or clone any person's facebook friendlist or followers accounts which have simple password

Muhammad Hamza 3 Dec 06, 2021
Domain abuse scanner covering domainsquatting and phishing keywords.

🦷 monodon 🐋 Domain abuse scanner covering domainsquatting and phishing keywords. Setup Monodon is a Python 3.7+ programm. To setup on a Linux machin

2 Mar 15, 2022
Northwave Log4j CVE-2021-44228 checker

Northwave Log4j CVE-2021-44228 checker Friday 10 December 2021 a new Proof-of-Concept 1 addressing a Remote code Execution (RCE) vulnerability in the

Northwave 125 Dec 09, 2022
Search Shodan for Minecraft server IPs to grief

GriefBuddy This script searches Shodan for Minecraft server IPs to grief. This will return all servers connected to the public internet which Shodan h

26 Dec 29, 2022
Springboot directory scanning

Springboot directory scanning

WINEZERO 87 Dec 28, 2022
Script Crack Facebook Premium 🚶‍♂

prem Script Crack Facebook Premium 🚶‍♂ Install Script $ pkg update && pkg update $ termux-setup-storage $ pkg install git $ pkg install python $ pip

Yumasaa 1 Dec 03, 2021
Extensive Python3 network scanner, simplified.

Snake Map Extensive Python3 network scanner, simplified. _,.--. --..,_ .'`__ o `;__, `'.'. .'.'` '---'` '

Miss Bliss 4 Apr 16, 2022
Python implementation of the diceware password generating algorithm.

Diceware Password Generator - Generate High Entropy Passwords Please Note - This Program Do Not Store Passwords In Any Form And All The Passwords Are

Sameera Madushan 35 Dec 25, 2022
CVE-2021-22205& GitLab CE/EE RCE

Vuln Impact An issue has been discovered in GitLab CE/EE affecting all versions starting from 11.9. GitLab was not properly validating image files tha

Al1ex 213 Dec 30, 2022
一个自动挖掘漏洞的框架,日后会发展成强大的信息收集+漏洞挖掘脚本!

介绍 工具介绍 这是一款致力于将各类优秀脚本集合在一起调用、联动,最终可形成超级渗透脚本的工具。目的是扫描到更全的资产信息,发现更多的漏洞利用。但是这是通过牺牲扫描速度来提升扫描广度的。所以不太适合要进行紧急信息收集和漏洞利用的情况。

Thinking rookie 23 Jul 05, 2022
Lite version of my Gatekeeper backdoor for public use.

MayorSec Backdoor Fully functioning bind-type backdoor This backdoor is a fully functioning bind shell and lite version of my full functioning Gatekee

Joe Helle 56 Mar 25, 2022
🍉一款基于Python-Django的多功能Web安全渗透测试工具,包含漏洞扫描,端口扫描,指纹识别,目录扫描,旁站扫描,域名扫描等功能。

Sec-Tools 项目介绍 系统简介 本项目命名为Sec-Tools,是一款基于 Python-Django 的在线多功能 Web 应用渗透测试系统,包含漏洞检测、目录识别、端口扫描、指纹识别、域名探测、旁站探测、信息泄露检测等功能。本系统通过旁站探测和域名探测功能对待检测网站进行资产收集,通过端

简简 300 Jan 07, 2023
Generate malicious files using recently published homoglyphic-attack (CVE-2021-42694)

CVE-2021-42694 Generate malicious files using recently published homoglyph-attack vulnerability, which was discovered at least in C, C++, C#, Go, Pyth

js-on 17 Dec 11, 2022
Credit Card And SK Checker Written In Python

💳 Credit Card Checker (CC Checker) & Mass SK Checker & Generator 💳

Rimuru Tempest 53 Dec 31, 2022