A high-performance DNS stub resolver for bulk lookups and reconnaissance (subdomain enumeration)

Overview

MassDNS

A high-performance DNS stub resolver

MassDNS is a simple high-performance DNS stub resolver targeting those who seek to resolve a massive amount of domain names in the order of millions or even billions. Without special configuration, MassDNS is capable of resolving over 350,000 names per second using publicly available resolvers.

Contributors

Compilation

Clone the git repository and cd into the project root folder. Then run make to build from source. If you are not on Linux, run make nolinux. On Windows, the Cygwin packages gcc-core, git and make are required.

Usage

Usage: ./bin/massdns [options] [domainlist]
  -b  --bindto           Bind to IP address and port. (Default: 0.0.0.0:0)
      --busy-poll        Use busy-wait polling instead of epoll.
  -c  --resolve-count    Number of resolves for a name before giving up. (Default: 50)
      --drop-group       Group to drop privileges to when running as root. (Default: nogroup)
      --drop-user        User to drop privileges to when running as root. (Default: nobody)
      --extended-input   Input names are followed by a space-separated list of resolvers.
                         These are used before falling back to the resolvers file.
      --filter           Only output packets with the specified response code.
      --flush            Flush the output file whenever a response was received.
  -h  --help             Show this help.
      --ignore           Do not output packets with the specified response code.
  -i  --interval         Interval in milliseconds to wait between multiple resolves of the same
                         domain. (Default: 500)
  -l  --error-log        Error log file path. (Default: /dev/stderr)
      --norecurse        Use non-recursive queries. Useful for DNS cache snooping.
  -o  --output           Flags for output formatting.
      --predictable      Use resolvers incrementally. Useful for resolver tests.
      --processes        Number of processes to be used for resolving. (Default: 1)
  -q  --quiet            Quiet mode.
      --rand-src-ipv6    Use a random IPv6 address from the specified subnet for each query.
      --rcvbuf           Size of the receive buffer in bytes.
      --retry            Unacceptable DNS response codes.
                         (Default: All codes but NOERROR or NXDOMAIN)
  -r  --resolvers        Text file containing DNS resolvers.
      --root             Do not drop privileges when running as root. Not recommended.
  -s  --hashmap-size     Number of concurrent lookups. (Default: 10000)
      --sndbuf           Size of the send buffer in bytes.
      --status-format    Format for real-time status updates, json or ansi (Default: ansi)
      --sticky           Do not switch the resolver when retrying.
      --socket-count     Socket count per process. (Default: 1)
  -t  --type             Record type to be resolved. (Default: A)
      --verify-ip        Verify IP addresses of incoming replies.
  -w  --outfile          Write to the specified output file instead of standard output.

Output flags:
  L - domain list output
  S - simple text output
  F - full text output
  B - binary output
  J - ndjson output

Advanced flags for the domain list output mode:
  0 - Include NOERROR replies without answers.

Advanced flags for the simple output mode:
  d - Include records from the additional section.
  i - Indent any reply record.
  l - Separate replies using a line feed.
  m - Only output reply records that match the question name.
  n - Include records from the answer section.
  q - Print the question.
  r - Print the question with resolver IP address, Unix timestamp and return code prepended.
  s - Separate packet sections using a line feed.
  t - Include TTL and record class within the output.
  u - Include records from the authority section.

Advanced flags for the ndjson output mode:
  e - Write a record for each terminal query failure.

For a detailed description of the command line interface, please consult the man page using man ./doc/massdns.1.

Example

Resolve all AAAA records from domains within domains.txt using the resolvers within resolvers.txt in lists and store the results within results.txt:

$ ./bin/massdns -r lists/resolvers.txt -t AAAA domains.txt > results.txt

This is equivalent to:

$ ./bin/massdns -r lists/resolvers.txt -t AAAA -w results.txt domains.txt

Example output

By default, MassDNS will output response packets in text format which looks similar to the following:

;; Server: 77.41.229.2:53
;; Size: 93
;; Unix time: 1513458347
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 51298
;; flags: qr rd ra ; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 0

;; QUESTION SECTION:
example.com. IN A

;; ANSWER SECTION:
example.com. 45929 IN A 93.184.216.34

;; AUTHORITY SECTION:
example.com. 24852 IN NS b.iana-servers.net.
example.com. 24852 IN NS a.iana-servers.net.

The resolver IP address is included in order to make it easier for you to filter the output in case you detect that some resolvers produce bad results.

Resolving

The repository includes the file resolvers.txt consisting of a filtered subset of the resolvers provided by the subbrute project. Please note that the usage of MassDNS may cause a significant load on the used resolvers and result in abuse complaints being sent to your ISP. Also note that the provided resolvers are not guaranteed to be trustworthy. The resolver list is currently outdated with a large share of resolvers being dysfunctional.

MassDNS's DNS implementation is currently very sporadic and only supports the most common records. You are welcome to help changing this by collaborating.

PTR records

MassDNS includes a Python script allowing you to resolve all IPv4 PTR records by printing their respective queries to the standard output.

$ ./scripts/ptr.py | ./bin/massdns -r lists/resolvers.txt -t PTR -w ptr.txt

Please note that the labels within in-addr.arpa are reversed. In order to resolve the domain name of 1.2.3.4, MassDNS expects 4.3.2.1.in-addr.arpa as input query name. As a consequence, the Python script does not resolve the records in an ascending order which is an advantage because sudden heavy spikes at the name servers of IPv4 subnets are avoided.

Reconnaissance by brute-forcing subdomains

Perform reconnaissance scans responsibly and adjust the -s parameter to not overwhelm authoritative name servers.

Similar to subbrute, MassDNS allows you to brute force subdomains using the included subbrute.py script:

$ ./scripts/subbrute.py lists/names.txt example.com | ./bin/massdns -r lists/resolvers.txt -t A -o S -w results.txt

As an additional method of reconnaissance, the ct.py script extracts subdomains from certificate transparency logs by scraping the data from crt.sh:

$ ./scripts/ct.py example.com | ./bin/massdns -r lists/resolvers.txt -t A -o S -w results.txt

The files names.txt and names_small.txt, which have been copied from the subbrute project, contain names of commonly used subdomains. Also consider using Jason Haddix' subdomain compilation with over 1,000,000 names.

Screenshots

Screenshot

Security

MassDNS does not require root privileges and will therefore drop privileges to the user called "nobody" by default when being run as root. If the user "nobody" does not exist, MassDNS will refuse execution. In this case, it is recommended to run MassDNS as another non-privileged user. The privilege drop can be circumvented using the --root argument which is not recommended. Also note that other branches than master should not be used in production at all.

Practical considerations

Performance tuning

MassDNS is a simple single-threaded application designed for scenarios in which the network is the bottleneck. It is designed to be run on servers with high upload and download bandwidths. Internally, MassDNS makes use of a hash map which controls the concurrency of lookups. Setting the size parameter -s hence allows you to control the lookup rate. If you are experiencing performance issues, try adjusting the -s parameter in order to obtain a better success rate.

Rate limiting evasion

In case rate limiting by IPv6 resolvers is a problem, have a look at the freebind project including packetrand, which will cause each packet to be sent from a different IPv6 address from a routed prefix.

Result authenticity

If the authenticity of results is highly essential, you should not rely on the included resolver list. Instead, set up a local unbound resolver and supply MassDNS with its IP address. In case you are using MassDNS as a reconnaissance tool, you may wish to run it with the default resolver list first and re-run it on the found names with a list of trusted resolvers in order to eliminate false positives.

In case you are enumerating subdomains for a single name, e.g. for example.com, you may want to simply leave out third-party resolvers. In this case, you can directly probe the authoritative nameservers like so:

$ ./bin/massdns -r <(./scripts/auth-addrs.sh example.com) --norecurse -o Je example-com-subdomains.txt > results.txt

Todo

  • Prevent flooding resolvers which are employing rate limits or refusing resolves after some time
  • Implement bandwidth limits
  • Employ cross-resolver checks to detect DNS poisoning and DNS spam (e.g. Level 3 DNS hijacking)
  • Add wildcard detection for reconnaissance
  • Improve reconnaissance reliability by adding a mode which re-resolves found domains through a list of trusted (local) resolvers in order to eliminate false positives
  • Detect optimal concurrency automatically
  • Parse the command line properly and allow the usage/combination of short options without spaces
Comments
  • many fake records

    many fake records

    Hi,

    I was running your tool on my vps , it was running fine for quite sometime but yesterday I installed it on new VPS and I am getting weird result. there is one weird server(142.54.173.92) which is sending fake records , this server does not listed on resolvers.txt(using from your repo). It looks like something is wrong. I have tried to traceroute this ip but last 10 hopes are not visible and i have also tried to see code massdns but it is in compiled format.

    I have copied your repo from my local system to vps wihtout using make command and once i received weird result , i delete entire folder and clone from online using git clone command and use make command .......still i receive this weird records from this weird server ...... It looks like two possible scenario , one of the resolver from list is sending records to this fake dns server(142.54.173.92) which is sending fake records as result. second could be somewhere in my vps , this records are stored due to using existing copy of the folder and re installation is not removing it .....i grep dns but i could not find any susceptible file

    ./subbrute.py names.txt example.com | ./bin/massdns -r resolvers.txt -t A -a -o -c 100 -w 18_dec_1.txt -

    [18_dec_1.txt]

    Note : i have changed result to example.com in above command and output file attached. I really appreciate if you can check on your side that if you are receiving this server in output or something wrong with massdns compiled file or can you provide me any lead to resolve this issue ?

    Thanks, Anis

    opened by yam23 16
  • Installation error

    Installation error

    HI,

    I am getting the error below while compiling the file after cloning from git. Please help.

    $ make mkdir -p bin cc -O3 -std=c11 -Wall -fstack-protector-strong *.c -o bin/massdns -ldl In file included from main.c:7: ./massdns.h:7:10: fatal error: 'sys/epoll.h' file not found #include <sys/epoll.h> ^~~~~~~~~~~~~ 1 error generated. make: *** [all] Error 1

    opened by h1hakz 12
  • Emtpy results.txt MACOSX

    Emtpy results.txt MACOSX

    Hello,

    I installed massdns via brew in MacosX. It installed successfully, but when i run any command the results.txt is empty. I have tried the --root option as well.

    massdns -r resolvers.txt -t AAAA -w results.txt urls.txt massdns -r resolvers.txt -t AAAA -w results.txt urls.txt --root massdns -r resolvers.txt -t AAAA -w results.txt urls.txt --root --error-log error.txt --busy-poll

    resolvers.txt contains only 8.8.8.8 urls.txt contains only google.com

    OUTPUT:

    Processed queries: 1 Received packets: 0 Progress: 100.00% (00 h 00 min 25 sec / 00 h 00 min 25 sec) Current incoming rate: 0 pps, average: 0 pps Current success rate: 0 pps, average: 0 pps Finished total: 1, success: 0 (0.00%) Mismatched domains: 0 (0.00%), IDs: 0 (0.00%) Failures: 0: 0.00%, 1: 0.00%, 2: 0.00%, 3: 0.00%, 4: 0.00%, 5: 0.00%, 6: 0.00%, 7: 0.00%, 8: 0.00%, 9: 0.00%, 10: 0.00%, 11: 0.00%, 12: 0.00%, 13: 0.00%, 14: 0.00%, 15: 0.00%, 16: 0.00%, 17: 0.00%, 18: 0.00%, 19: 0.00%, 20: 0.00%, 21: 0.00%, 22: 0.00%, 23: 0.00%, 24: 0.00%, 25: 0.00%, 26: 0.00%, 27: 0.00%, 28: 0.00%, 29: 0.00%, 30: 0.00%, 31: 0.00%, 32: 0.00%, 33: 0.00%, 34: 0.00%, 35: 0.00%, 36: 0.00%, 37: 0.00%, 38: 0.00%, 39: 0.00%, 40: 0.00%, 41: 0.00%, 42: 0.00%, 43: 0.00%, 44: 0.00%, 45: 0.00%, 46: 0.00%, 47: 0.00%, 48: 0.00%, 49: 0.00%, 50: 100.00%, Response: | Success: | Total: OK: | 0 ( 0.00%) | 0 ( 0.00%) NXDOMAIN: | 0 ( 0.00%) | 0 ( 0.00%) SERVFAIL: | 0 ( 0.00%) | 0 ( 0.00%) REFUSED: | 0 ( 0.00%) | 0 ( 0.00%) FORMERR: | 0 ( 0.00%) | 0 ( 0.00%)

    opened by Ncoder2 9
  • Limits can slow massdns?

    Limits can slow massdns?

    Hello! Run as root --root -s 5000, but still uses about 40 mbps (half of the channel). Virtualbox ubuntu 16.04: 64 bit, 12gb ram 3 core, Connection PPPoe.

    Maybe problem with limits: ?

    [email protected]:~# ulimit -a
    core file size          (blocks, -c) 0
    data seg size           (kbytes, -d) unlimited
    scheduling priority             (-e) 0
    file size               (blocks, -f) unlimited
    pending signals                 (-i) 64040
    max locked memory       (kbytes, -l) 64
    max memory size         (kbytes, -m) unlimited
    open files                      (-n) 100000
    pipe size            (512 bytes, -p) 8
    POSIX message queues     (bytes, -q) 819200
    real-time priority              (-r) 0
    stack size              (kbytes, -s) 8192
    cpu time               (seconds, -t) unlimited
    max user processes              (-u) 64040
    virtual memory          (kbytes, -v) unlimited
    file locks                      (-x) unlimited
    

    Cheers,

    opened by Roundik 6
  • Possible Encoding Bug

    Possible Encoding Bug

    I am using massdns compiled on this commit:

    d8b911d26fe6af80ef2d2647220e9a48ac832d3f

    I am using ndjson output mode. And when I run massdns for RRSIG record type, I get this value. Domain name is swordsec.com

     {'asset_name_or_value':
     {'fqdn': 'swordsec.com.',
     'record_type': 'RRSIG',
     'value': '.\\x0d\\x02\\x00\\x00\\x0e\\xcd`\\x1e\\x96\\x88`\\x1b\\xd7.\\x86\\xc9\\x08swordsec\\x03com\\x002\\xec\\xad\\x05\\x140\\x06\\x12\\x9c\\x07s\\xd3\\x89v\\xdf\\xabI^\\xb6\\x0at\\x8a\\xe1\\xa0\\xc691\\x9a\\x06\\x90\\x83\\xf5\\xb0\\x13\\xc4d\\x9d\\x99\\x05+\\x9c4ei\\x0e \\xb0S\\x99\\x8f\\xd4]f\\xc3\\xcc\\xc2W_\\xf6!\\xbdA\\x01\\xdb',
     'resolver': '64.6.64.6:53',
     'status': 'NOERROR'},
     time_now': '2021-02-05 12:15:54.256611'}
    
    

    As you can see there is "swordsec\x03com" in value, so I guess the correct encoding would result "swordsec.com" in that part. I think there might be a encoding error. I also noticed similar encoding in DNSKEY and CDNSKEY records too.

     {'asset_name_or_value': 
    {'fqdn': 'swordsec.com.',
     'record_type': 'DNSKEY',
     'value': '\\x00.\\x0d\\xa0\\x93.\\x11,\\xf9\\x13\\x88\\x18\\xcd/\\xea\\xe9p\\xeb\\xbdMj0\\xf6.\\x8c%\\xb3%\\xa3\\x9a\\xbb\\xc5.\\x11\\x97\\xaa\\x09\\x82\\x83\\xe5\\xaa\\xf4!\\x17|*\\xa5\\xd7\\x14\\x99*\\x99W\\xd1\\xbc\\xc1\\x8f\\x98\\xcdq\\xf1\\xf1\\x80ke\\xe1H',
     'resolver': '77.88.8.8:53',
     'status': 'NOERROR'},
     ''time_now': '2021-02-05 12:15:54.295338'}
    
    {'asset_name_or_value': {
    'fqdn': 'swordsec.com.',
     'record_type': 'CDNSKEY',
     'value':"\\x01.\\x0d\\x99\\xdb.\\xc1L\\xab\\xdc3\\xd6\\xd7}\\xa6:/\\x15\\xf7\\x11\\x12XO#N\\x8d\\x1d\\xc4(\\xe3\\x9e\\x8aJ\\x97\\xe1\\xaa'\\x1aU]\\xc9\\x07\\x01\\xe1~*LKo\\x12.|2\\xd4OJ\\xc0+\\xd8\\x94\\xcf-.\\xe7w\\x8a\\x19", 
    'resolver': '8.8.4.4:53', 
    'status': 'NOERROR'},
    'time_now': '2021-02-05 12:15:54.268398'}
    
    
    opened by srknzl 4
  • Poor performance on Digital Ocean droplet

    Poor performance on Digital Ocean droplet

    Massdns used to work perfectly for at least 1 year on Digital Ocean Droplets (DOD) with performance ~22k pps. Recently after I apt update && apt upgrade my Ubuntu machine on DOD the massdns became unusable with performance ~50-300 pps. Now massdns often completely freezes SSH connection so hard reset via DOD cPanel is needed. It has never happened before.

    Currently I'm using massdns from my home connection (150Mb/s) with the same public resolvers which I used at DOD and I'm getting ~8-10k pps. I tried to create fresh DOD and clean install massdns but it didn't help. I even opened support ticket a explained the problem to the DO support but got response that all networking regarding my DOD is fully functional.

    Does anyone have similar issues on latest Ubuntu machines or/and DODs?

    opened by MMquant 4
  • Feedback & list of issues I am facing

    Feedback & list of issues I am facing

    Greetings!! Thanks for this wonderful tool. I am facing following issues while using this tool:

    1. The tool is not scanning all domains (when given list of 10000 domains hardly scans 1000 of them which I verified by grep "QUESTION SECTION" in output file generated with -o F switch). Although above problem is solved for now by using -s 1000(default is 10000) so I think you should change the default value to 1000.
    2. No explicit option to grep Domain having NXDOMAIN status since it is required for finding domains vulnerable to subdomain takeover(like Azure). Although above problem can be solved by : ./massdns -q -t A -r /opt/resolvers.txt domain_list.txt -o F -w output_massdns.txt cat output_massdns.txt | grep NXDOMAIN -A 7 | grep CNAME But it is not efficient.
    3. Getting different result when using -o F and then -w and only using -w (without any output flag) Although result should come same.So please tell me if -o F append some extra details which are drop when we are not using it.
    4. Include Option which do below operation: Take list of domains and output only those domain names whose IP can be resolved. (Should also exclude those domains which have status NO ERROR but only have CNAME and no IP) Since this list can be passed further to httprobe(or CURL) or other tools which do some active scanning.(As list fetched above will assure that the domains' IP recored exist).
    opened by mansoorr123 4
  • Benchmark on ZDNS with MassDNS?

    Benchmark on ZDNS with MassDNS?

    ZDNS was another tool to lookup dns records which was developed by ZMap Team. Have you ever do some comparison on ZMap with MassDNS. I wonder which is better for subdomain enumerations.

    opened by jkryanchou 4
  • Massdns does not respect concurrent lookups parameter

    Massdns does not respect concurrent lookups parameter

    Hi there,

    When setting a low number of concurrent lookups with a short list of resolvers, massdns doesn't cap the concurrent lookups and use a much higher throughput.

    For example, the following command : massdns -t A -s 10 -r resolvers.txt -o S -w output.txt domains.txt

    with a "resolvers.txt" file containing 1 resolver will execute around 500 concurrent lookups. The same command with a "resolvers.txt" file containing about 30 resolvers will execute around 150 concurrent lookups.

    If I increase the --hashmap-size number, the multiplication factor remains the same. If I use the whole list of resolvers from https://public-dns.info/, the number of concurrent lookups I've set is respected.

    I suspect this is related to how the hashmap works and I'm not sure it's easily fixable.

    opened by h49nakxs 3
  • Add epoch timestamps to responses in NDJSON output format (app start only, rx only, or rx+tx)

    Add epoch timestamps to responses in NDJSON output format (app start only, rx only, or rx+tx)

    I have a workflow where I often have batches of massdns NDJSON files and batches of data from various other tools. The challenge I'm having right now is ensuring that the most recent DNS responses are used when merging/joining these datasets when there are duplicate questions with different answers- due to a change in the zone file that occurred some time between the two sessions that generated the batches. In some cases, massdns NDJSON files have been concatenated, or don't have accurate mtime/ctime. So I'm kind of helpless there

    The simple solution to this is in line with what other similar tools do- stamp an epoch timestamp into the record

    For my purposes, it would be a huge help even if it was just a fixed time that was retrieved once and stamped into every NDJSON row, grabbed with time() at startup

    The most correct and/or powerful/broadly useful implementation would probably be to have both a tx and rx stamp, as two logically separate fields. Something like:

    {
      "name": "www.bah.com.",
      "type": "A",
      "class": "IN",
      "status": "NOERROR",
      "tx_ts": 1614783271,
      "rx_ts": 1614783273,
      "data": {
        "answers": [
          {
            "ttl": 300,
            "type": "A",
            "class": "IN",
            "name": www.bah.com,
            "data": "1.2.3.4"
          }
        ],
      },
      "resolver": "1.2.3.4:53"
    }
    

    I realize that capturing the transmit stamp may require thoughtfulness to avoid impacting the transmission rate, and I also need to re-familiarize myself with the massdns code before saying this will be simple. But I put a little bit of thought into it as I was writing this up.

    A naive implementation would be continuously calling time(), which would result in a continuous flood of system calls. Not really a good solution and may have a significant performance impact. That's not what I would like to do.

    I'm guessing that the best solution would be one that avoids system calls/interrupts entirely, either by directly making use of architecture-specific instructions (e.g. rdtsc for x86/x86_64) or a libc wrapper for the same. I think (need to check) that some or all of the "clock" types in glibc clock_gettime() utilize architecture-specific instructions instead of system calls. I think most (if not all) are accessible from userspace. FWIW, I have access to ARM and PPC64 machines to check the portability of whatever seems to be a good solution, to avoid breaking people on weird architectures

    I don't expect any CPU instructions to return epoch time, but if epoch time is captured once at the start of the application, the ticks can just scaled (from nanoseconds, milliseconds, or whatever) to seconds and then subtracted from that initial epoch value without any measurable impact to performance

    I think the choices are (in order of least effort, least invasiveness to most):

    1. This is a dumb idea. Do nothing
    2. Add a single static timestamp to every NDJSON row, representing the time the process started
    3. Add a dynamically retrieved timestamp to every NDJSON row, generated at the time the response is received
    4. Add both a transmit and receive timestamp to every NDJSON row, generated before transmit and after receive

    I think #4 is doable. #2 solves my problem "well enough". I'm happy to take a shot at any of them.

    Thoughts on this?

    As always, I appreciate the time you put into this, it's a very useful tool for quite a few workflows I have

    (BTW- I'll feel very silly if this data is already captured and just not stamped into the row- I just didn't have time to look through the source before entering this)

    opened by mzpqnxow 3
  • Errors when processing large file

    Errors when processing large file

    I'm struggling to find the cause of two errors that I get when I try to run massdns on a file of generated subdomains. Most of the subdomains are being resolved but I get the error "Error sending: Bad address" and "Error sending: Address family not supported by protocol" seemingly randomly. I've changed the hashmap-size and interval but still get this error. The subdomains being resolved all appear to be valid and were generated with alt-dns. Is there anyway to find out which subdomains exactly are causing this error?

    I've also tried running it on a Ubuntu VM and Windows, both successfully compile and run, but give the same errors.

    opened by yr-or 3
  • Epoll failure on interrupted system call

    Epoll failure on interrupted system call

    I ran massdns for several hours to resolve large number of domains, and it quit prematurely with

    DEBUG Epoll failure: Interrupted system call
    

    This comes from https://github.com/blechschmidt/massdns/blob/master/src/main.c#L2013.

    I'm not expert on this, but maybe epoll_wait() failures due to signal interrupts could be ignored?

    At least that the understanding I got from e.g. https://stackoverflow.com/a/6870391

    opened by suola 3
  • No output stdout of servfails

    No output stdout of servfails

    I am trying to understand why massdns it isn't giving any output to stdout.

    I run the same exact command in one server and it runs correctly, but running it in another server gives no outout to stdout (I think it is because the version of massdns).

    I can't get massdns to print me servfail results.

    echo servfail-domain.com | massdns -r google -s 2000 --flush -o J -c 5 2> /dev/null gives no output, while in my other server I get a json containing the servfail domain.

    {"name":...

    edit: Just confirmed that it is because it was an old version of massdns. I was able to reproduce by getting the executable to the new vps, the last version of massdns won't output anythng. Any idea why it isn't working anymore in newest versions?

    opened by marcelo321 1
  • Surprising high number of lost lookups: about 20% false negatives

    Surprising high number of lost lookups: about 20% false negatives

    Hey there,

    I noticed quite a lot of lost lookups with massdns. In the example below, it roughly misses about 1900 names out of 10k domains.

    I am not sure what these false negatives come from though, apologies for not trying to find where the bug is in massdns.

    I made a screen recording https://asciinema.org/a/415235 of the reproduction steps below:

    $ docker run --rm -it --entrypoint sh blechschmidt/massdns
    /massdns # apk --no-cache add git alpine-sdk curl ipython py3-dnspython
    /massdns # git pull
    /massdns # make
    /massdns # cat > /resolvers.txt <<EOF # See https://gist.github.com/seb-elttam/af28008b092eb5bcdfede8565c55147e#file-mtresolver-py-L18
    1.1.1.1
    1.0.0.1
    8.8.8.8
    8.8.4.4
    9.9.9.10
    149.112.112.10
    94.140.14.140
    94.140.14.141
    64.6.64.6
    64.6.65.6
    77.88.8.8
    77.88.8.1
    74.82.42.42
    EOF
    
    /massdns # curl -s http://s3.amazonaws.com/alexa-static/top-1m.csv.zip | unzip -p - | cut -d, -f2- \
     | head -n 10k  > /domains.txt
    
    /massdns # time ./bin/massdns -s 10000 -q -r /resolvers.txt -o S -w /out.txt /domains.txt; \
     cat /out.txt | grep ' A [0-9]' | cut -d' ' -f1 | sort -u | wc -l
    real    0m 7.07s
    user    0m 0.09s
    sys     0m 0.21s
    7534
    
    /massdns # time ./bin/massdns -s 1000 -q -r /resolvers.txt -o S -w /out.txt /domains.txt; \
     cat /out.txt | grep ' A [0-9]' | cut -d' ' -f1 | sort -u | wc -l
    real	0m 5.03s
    user	0m 0.10s
    sys	0m 0.18s
    7911
    
    /massdns # git clone https://gist.github.com/seb-elttam/af28008b092eb5bcdfede8565c55147e /gist && cd /gist
    /gist # ipython
    Python 3.9.5 (default, May 12 2021, 20:44:22) 
    Type 'copyright', 'credits' or 'license' for more information
    IPython 7.23.1 -- An enhanced Interactive Python. Type '?' for help.
    
    In [1]: from mtresolver import *
       ...: r = resolve_hostnames('/domains.txt', 1000)
       ...: len(r)
    06:38:03 DEBUG enter
    06:38:21 DEBUG exit
    Out[1]: 9847
    In [2]: from mtresolver import *
       ...: r = resolve_hostnames('/domains.txt', 10000)
       ...: len(r)
    06:39:07 DEBUG enter
    06:39:25 DEBUG exit
    Out[2]: 9850
    
    opened by ghost 3
  • Trying to understand the output

    Trying to understand the output

    Hi,

    Thanks for this cool program :) I was trying to do a "dig" for each domain manually, and it was bloomin slow. So I'm using:

    ./massdns/bin/massdns -r massdns/lists/resolvers.txt -o S -t A massdns/domains-to-do.txt > results.txt

    Which finishes with:

    Processed queries: 239005
    Received packets: 405301
    Progress: 100.00% (00 h 00 min 57 sec / 00 h 00 min 57 sec)
    Current incoming rate: 24 pps, average: 7054 pps
    Current success rate: 1 pps, average: 4160 pps
    Finished total: 239004, success: 239002 (100.00%)
    Mismatched domains: 17138 (4.29%), IDs: 0 (0.00%)
    Failures: 0: 19.28%, 1: 16.90%, 2: 13.38%, 3: 10.62%, 4: 8.31%, 5: 6.52%, 6: 5.10%, 7: 3.94%, 8: 3.12%, 9: 2.47%, 10: 2.01%, 11: 1.59%, 12: 1.27%, 13: 1.02%, 14: 0.82%, 15: 0.67%, 16: 0.57%, 17: 0.45%, 18: 0.35%, 19: 0.31%, 20: 0.24%, 21: 0.20%, 22: 0.15%, 23: 0.13%, 24: 0.10%, 25: 0.08%, 26: 0.06%, 27: 0.05%, 28: 0.05%, 29: 0.04%, 30: 0.04%, 31: 0.02%, 32: 0.02%, 33: 0.02%, 34: 0.01%, 35: 0.02%, 36: 0.01%, 37: 0.01%, 38: 0.01%, 39: 0.01%, 40: 0.01%, 41: 0.00%, 42: 0.00%, 43: 0.00%, 44: 0.00%, 45: 0.00%, 46: 0.00%, 47: 0.00%, 48: 0.00%, 49: 0.00%, 50: 0.00%,
    Response: | Success:               | Total:
    OK:       |       229527 ( 96.04%) |       242872 ( 60.73%)
    NXDOMAIN: |         1643 (  0.69%) |         1676 (  0.42%)
    SERVFAIL: |         7832 (  3.28%) |         9740 (  2.44%)
    REFUSED:  |            0 (  0.00%) |       145641 ( 36.42%)
    FORMERR:  |            0 (  0.00%) |            0 (  0.00%)
    

    My file has 239,005 domains in - but I see quite a lot of SERVEFAIL and NXDOMAIN errors (over 9k of them). What do those mean? I'm going to run a test to compare what is outputted, vs what I'm passing in - but the outputted results.txt file is only 229,641 lines, which makes me think quite a few are missing - especially when a lot of domains have multiple lines:

    aradhana.org.uk. A 185.230.63.107
    aradhana.org.uk. A 185.230.63.186
    aradhana.org.uk. A 185.230.63.171
    

    Thanks in advance!

    Andy

    opened by youradds 2
  • Different results in output -o S and -o J (simple and json)

    Different results in output -o S and -o J (simple and json)

    Hello,

    After 2 hours of testing I found out that i am getting different results from simple output that json output and it is getting me crazy.

    So i am trying to get NOERROR responses like:

    ...-o J 2> /dev/null | jq -r 'select(.status=="NOERROR") .name' and I get like 50 results. with only 6 being valid (only 6 of the total are valid).

    Running the same command but like -o S 2> /dev/null | awk '{print $1}' | anew and I get only the 6 valid, and the command is working as expected.

    Then I analyzed the json output and noticed that I am getting a looot of noerror responses that are incorrect, but for some reason that isn't happening with simple output.

    I made the tests against same hosts and with the same resolvers countless of times and getting really different responses.

    opened by marcelo321 3
  • No output created when using --ignore

    No output created when using --ignore

    cat hostfile | dnsgen - | massdns -r ~/tools/massdns-1.0.0/lists/resolvers.txt --ignore NXDOMAIN --ignore SERVFAIL --ignore REFUSED -t A -o S -w resolved-hosts

    This results in nothing being saved. All the OK should be in the output. massdns is running fine but nothing is being saved

    I located the error to the -o S option for simple output.

    opened by math-ematics 2
Releases(v1.0.0)
  • v1.0.0(Oct 8, 2021)

    With the release of version 1.0.0, MassDNS switches to semantic versioning. Compared to the previous release, response code filtering using --ignore and --filter is new. Additionally, a bug in the NDJSON output was fixed by commit 0b507651cce50e1e3d4073a7956fbc82c8fd8efb. Furthermore, the handling of unconventional DNS names was improved (23ee79a0a410ff326f23a44b83e934d77181e23c and related commits).

    Source code(tar.gz)
    Source code(zip)
  • v0.3(Mar 29, 2020)

    This version of MassDNS uses a custom DNS implementation in order to improve performance by minimizing memory allocations. For this reason, MassDNS no longer depends on ldns. In addition, MassDNS now supports outputting results as NDJSON.

    Source code(tar.gz)
    Source code(zip)
Owner
B. Blechschmidt
B. Blechschmidt
Safety checks your installed dependencies for known security vulnerabilities

Safety checks your installed dependencies for known security vulnerabilities. By default it uses the open Python vulnerability database Safety DB, but

pyup.io 1.4k Dec 30, 2022
Hadoop Yan ResourceManager unauthorized RCE

Vuln Impact There was an unauthorized access vulnerability in Hadoop yarn ResourceManager. This vulnerability existed in Hadoop yarn, the core compone

Al1ex 25 Nov 24, 2022
Spray365 is a password spraying tool that identifies valid credentials for Microsoft accounts (Office 365 / Azure AD).

What is Spray365? Spray365 is a password spraying tool that identifies valid credentials for Microsoft accounts (Office 365 / Azure AD). How is Spray3

Mark Hedrick 246 Dec 28, 2022
Exploiting CVE-2021-42278 and CVE-2021-42287

noPac Exploiting CVE-2021-42278 and CVE-2021-42287 原项目noPac在实现上可能有点问题,导致在本地没有打通,于是参考sam-the-admin项目进行修改。 使用 pip3 install -r requirements.txt # GetShel

W4ter 2 Jun 23, 2022
Evil-stalker - A simple tool written in python, it is so simple that it is based on google dorks

evil-stalker How to run First of all, you must install the necessary libraries.

rock3d 6 Nov 16, 2022
A Modified version of TCC's Osprey poc framework......

fierce-fish fierce-fish是由TCC(斗象能力中心)出品并维护的开源漏洞检测框架osprey的改写,去掉臃肿功能的精简版本poc框架 PS:真的用不惯其它臃肿的功能,不过作为一个收集漏洞poc && exp的框架还是非常不错的!!! osprey For beginners fr

lUc1f3r11 10 Dec 30, 2022
Virus-Builder - This tool will generate a virus that can only destroy Windows computer

Virus-Builder - This tool will generate a virus that can only destroy Windows computer. You can also configure to auto run in usb drive

Saad 16 Dec 30, 2022
SSLyze is a fast and powerful SSL/TLS scanning tool and Python library.

SSLyze SSLyze is a fast and powerful SSL/TLS scanning tool and Python library. SSLyze can analyze the SSL/TLS configuration of a server by connecting

Alban Diquet 2.8k Jan 03, 2023
Binary check tool to identify command injection and format string vulnerabilities in blackbox binaries

Binary check tool to identify command injection and format string vulnerabilities in blackbox binaries. Using xrefs to commonly injected and format string'd files, it will scan binaries faster than F

Christopher Roberts 3 Nov 16, 2021
A Proof-Of-Concept for the recently found CVE-2021-44228 vulnerability

log4j-shell-poc A Proof-Of-Concept for the recently found CVE-2021-44228 vulnerability. Recently there was a new vulnerability in log4j, a java loggin

koz 1.5k Jan 04, 2023
A high-performance DNS stub resolver for bulk lookups and reconnaissance (subdomain enumeration)

MassDNS A high-performance DNS stub resolver MassDNS is a simple high-performance DNS stub resolver targeting those who seek to resolve a massive amou

B. Blechschmidt 2.5k Jan 07, 2023
Gefilte Fish GMail filter creator

Gefilte Fish: GMail filter maker Gefilte Fish automates the creation of GMail filters. Use it like this: from gefilte import GefilteFish,

Ned Batchelder 31 Sep 28, 2022
PyExtractor is a decompiler that can fully decompile exe's compiled with pyinstaller or py2exe

PyExtractor is a decompiler that can fully decompile exe's compiled with pyinstaller or py2exe with additional features such as malware checker/detector! Also checks file(s) for suspicious words, dis

Rdimo 56 Jul 31, 2022
Growtopia Save.dat Stealer

savedat-stealer Growtopia Save.dat Stealer (Auto Send To Webhook) How To Use After Change Webhook URL Compile script to exe Give to target Done Info C

NumeX 9 May 01, 2022
D-810 is an IDA Pro plugin which can be used to deobfuscate code at decompilation time by modifying IDA Pro microcode.

Introduction fork from https://gitlab.com/eshard/d810 What is D-810 D-810 is an IDA Pro plugin which can be used to deobfuscate code at decompilation

Banny 30 Dec 06, 2022
xp_CAPTCHA(白嫖版) burp 验证码 识别 burp插件

xp_CAPTCHA(白嫖版) 说明 xp_CAPTCHA (白嫖版) 验证码识别 burp插件 安装 需要python3 小于3.7的版本 安装 muggle_ocr 模块(大概400M左右) python3 -m pip install -i http://mirrors.aliyun.com/

算命縖子 588 Jan 09, 2023
Scans all drives for log4j jar files and gets their version from the manifest

log4shell_scanner Scans all drives for log4j jar files and gets their version from the manifest. Windows and Windows Server only.

Zdeněk Loučka 1 Dec 29, 2021
This is tools hacking for scan vuln in port web, happy using

Xnuvers007 PortInjection this is tools hacking for scan vuln in port web, happy using view/show python 3.9 solo coder (tangerang) 19 y/o installation

XnuxersXploitXen 6 Dec 24, 2022
NoSecerets is a python script that is designed to crack hashes extremely fast. Faster even than Hashcat

NoSecerets NoSecerets is a python script that is designed to crack hashes extremely fast. Faster even than Hashcat How does it work? Instead of taking

DosentTrust GithubDatabase 9 Jul 04, 2022
A guide to building basic malware in Python by implementing a keylogger application

Keylogger-Malware-Project A guide to building basic malware in Python by implementing a keylogger application. If you want even more detail on the Pro

Noah Davis 1 Jan 11, 2022