Convert a collection of features to a fixed-dimensional matrix using the hashing trick.

Overview

FeatureHasher

Convert a collection of features to a fixed-dimensional matrix using the hashing trick.

Note, this requires Jina>=2.2.4.

Example

Here I use FeatureHasher to hash each sentence of Pride and Prejudice into a 128-dim vector, and then use .match to find top-K similar sentences.

from jina import Document, DocumentArray, Flow

# load 
   
d = Document(uri='https://www.gutenberg.org/files/1342/1342-0.txt').convert_uri_to_text()

# cut into non-empty sentences store in a DA
da = DocumentArray(Document(text=s.strip()) for s in d.text.split('\n') if s.strip())

# use FeatureHasher in a Flow
f = Flow().add(uses='jinahub://FeatureHasher')

embed_da = DocumentArray()
with f:
    f.post('/', da, on_done=lambda req: embed_da.extend(req.docs), show_progress=True)

print('self-matching...')
embed_da.match(embed_da, exclude_self=True, limit=5, normalization=(1, 0))
print('total sentences: ', len(embed_da))
for d in embed_da:
    print(d.text)
    for m in d.matches:
        print(m.scores['cosine'], m.text)
    input()
           [email protected][I]:🎉 Flow is ready to use!
	🔗 Protocol: 		GRPC
	🏠 Local access:	0.0.0.0:52628
	🔒 Private network:	192.168.178.31:52628
	🌐 Public address:	217.70.138.123:52628
⠹       DONE ━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0:00:01 100% ETA: 0 seconds 40 steps done in 1 second
total sentences:  12153
The Project Gutenberg eBook of Pride and Prejudice, by Jane Austen

   
     *** END OF THE PROJECT GUTENBERG EBOOK PRIDE AND PREJUDICE ***

    
      *** START OF THE PROJECT GUTENBERG EBOOK PRIDE AND PREJUDICE ***

     
       production, promotion and distribution of Project Gutenberg-tm

      
        Pride and Prejudice

       
         By Jane Austen This eBook is for the use of anyone anywhere in the United States and 
        
          This eBook is for the use of anyone anywhere in the United States and 
         
           by the awkwardness of the application, and at length wholly 
          
            Elizabeth passed the chief of the night in her sister’s room, and 
           
             the happiest memories in the world. Nothing of the past was 
            
              charities and charitable donations in all 50 states of the United 
            
           
          
         
        
       
      
     
    
   

In practice, you can implement matching and storing via an indexer inside Flow. This example is only for demo purpose so any non-feature hashing related ops are implemented outside the Flow to avoid distraction.

Owner
Jina AI
A Neural Search Company. We provide the cloud-native neural search solution powered by state-of-the-art AI technology.
Jina AI
Script hecho en python para sacar la informacion del numero de telefono, Hecha con el API de numverify

Script hecho en python para sacar la informacion del numero de telefono, Hecha con el API de numverify

DW Dariel 5 Dec 03, 2022
Metasploit Multi Purpose Exploiting Toolkit For Termux

MSF-EXPLOIT MSF-ANDRO is a Metasploit Multi Purpose Exploiting Toolkit For Termux . Only a Basic Script , Still in Development . FEATURES : Install Me

Mr.X 22 Dec 29, 2022
An easy-to-use wrapper for NTFS-3G on macOS

ezNTFS ezNTFS is an easy-to-use wrapper for NTFS-3G on macOS. ezNTFS can be used as a menu bar app, or via the CLI in the terminal. Installation To us

Matthew Go 34 Dec 01, 2022
Using python 3 and Flask an MVC system where the AES 128 CBC and Trivium algorithms

This project was developed using python 3 and Flask, it is an MVC system where the AES 128 CBC and Trivium algorithms can be tested through a communication between the computer and a device such as a

Brandon Israel Camacho Reyes 1 Dec 26, 2021
These are Simple python scripts to test/scan your network

Disclaimer This tool is for Educational purpose only. We do not promote or encourage any illegal activities. Summary These are Simple python scripts t

Varun Jagtap 5 Oct 08, 2022
Lightweight and beneficial Dependency Injection plugin for apscheduler

Implementation of dependency injection for apscheduler Prerequisites: apscheduler-di solves the problem since apscheduler doesn't support Dependency I

Glib 11 Dec 07, 2022
pwncat module that automatically exploits CVE-2021-4034 (pwnkit)

pwncat_pwnkit Introduction The purpose of this module is to attempt to exploit CVE-2021-4034 (pwnkit) on a target when using pwncat. There is no need

Dana Epp 33 Jul 01, 2022
A python script written for lazy people to hack their school systen ;D

F-ck-the-system A python script written for lazy people to hack their school systen ;D Python voice notes writer This is a python script to record you

Sachit 2 Jan 09, 2022
Yesitsme - Simple OSINT script to find Instagram profiles by name and e-mail/phone

Simple OSINT script to find Instagram profiles by name and e-mail/phone

108 Jan 07, 2023
spring-cloud-gateway-rce CVE-2022-22947

Spring Cloud Gateway Actuator API SpEL表达式注入命令执行(CVE-2022-22947) 1.installation pip3 install -r requirements.txt 2.Usage $ python3 spring-cloud-gateway

k3rwin 10 Sep 28, 2022
ProxyLogon(CVE-2021-26855+CVE-2021-27065) Exchange Server RCE(SSRF->GetWebShell)

ProxyLogon For Python3 ProxyLogon(CVE-2021-26855+CVE-2021-27065) Exchange Server RCE(SSRF-GetWebShell) usage: python ProxyLogon.py --host=exchang

112 Dec 01, 2022
macOS Initial Access Payload Generator

Mystikal macOS Initial Access Payload Generator Related Blog Post: https://posts.specterops.io/introducing-mystikal-4fbd2f7ae520 Usage: Install Xcode

Leo Pitt 206 Dec 31, 2022
Python tool for enumerating directories and for fuzzing

Python tool for enumerating directories and for fuzzing

Gourab Roy 5 Feb 21, 2022
neo Tool is great one in binary exploitation topic

neo Tool is great one in binary exploitation topic. instead of doing several missions by many tools and windows, you can now automate this in one tool in one session.. Enjoy it

Hamza Elansari 4 Oct 10, 2022
Writing and posting code throughout my new journey into python!

bootleg-productions consider this account to be a journal for me to record my progress throughout my python journey feel free to copy codes from this

1 Dec 30, 2021
Searches through git repositories for high entropy strings and secrets, digging deep into commit history

truffleHog Searches through git repositories for secrets, digging deep into commit history and branches. This is effective at finding secrets accident

Truffle Security 10.1k Jan 09, 2023
Hack computer in the form of RAR files from all types of clients, even Linux

Program Features 📌 Hide malware 📌 Vulnerability software vulnerabilities RAR 📌 Creating malware 📌 Access client files 📌 Client Hacking 📌 Link Do

hack4lx 5 Nov 25, 2022
It's a simple tool for test vulnerability Apache Path Traversal

SimplesApachePathTraversal Simples Apache Path Traversal It's a simple tool for test vulnerability Apache Path Traversal https://blog.mrcl0wn.com/2021

Mr. Cl0wn - H4ck1ng C0d3r 56 Dec 27, 2022
Cve-2021-22005-exp

cve-2021-22005-exp 0x01 漏洞简介 2021年9月21日,VMware发布安全公告,公开披露了vCenter Server中的19个安全漏洞,这些漏洞的CVSSv3评分范围为4.3-9.8。 其中,最为严重的漏洞为vCenter Server 中的任意文件上传漏洞(CVE-20

Jing Ling 146 Dec 31, 2022
automatically crawl every URL and find cross site scripting (XSS)

scancss Fastest tool to find XSS. scancss is a fastest tool to detect Cross Site scripting (XSS) automatically and it's also an intelligent payload ge

Md. Nur habib 30 Sep 24, 2022