A desktop GUI providing an audio interface for GPT3.

Overview

Jabberwocky

neil_degrasse_tyson_with_audio.mp4

Project Description

This GUI provides an audio interface to GPT-3. My main goal was to provide a convenient way to interact with various experts or public figures: imagine discussing physics with Einstein or hip hop with Kanye (or hip hop with Einstein? πŸ€” ). I often find writing and speaking to be wildly different experiences and I imagined the same would be true when interacting with GPT-3. This turned out to be partially true - the default Mac text-to-speech functionality I'm using here is certainly not as lifelike as I'd like. Perhaps more powerful audio generation methods will pop up in a future release...

We also provide Task Mode containing built-in prompts for a number of sample tasks:

  • Summarization
  • Explain like I'm 5
  • Translation
  • How To (step by step instructions for performing everyday tasks)
  • Writing Style Analysis
  • Explain machine learning concepts in simple language
  • Generate ML paper abstracts
  • MMA Fight Analysis and Prediction

Getting Started

  1. Clone the repo.
git clone https://github.com/hdmamin/jabberwocky.git
  1. Install the necessary packages. I recommend using a virtual environment of some kind (virtualenv, conda, etc). If you're not using Mac OS, you could try installing portaudio with whatever package manager you're using, but app behavior on other systems is unknown.
brew install portaudio
pip install -r requirements.txt
python -m nltk.downloader punkt

If you have make installed you can simply use the command:

make install
  1. Add your openai API key somewhere the program can access it. There are two ways to do this:
echo your_openai_api_key > ~/.openai

or

export OPENAI_API_KEY=your_openai_api_key

(Make sure to use your actual key, not the literal text your_openai_api_key.)

  1. Run the app.
python gui/main.py

Or with make:

make run

Usage

Conversation Mode

In conversation mode, you can chat with a number of pre-defined personas or add new ones. New personas can be autogenerated or defined manually.

See data/conversation_personas for examples of autogenerated prompts. You can likely achieve better results using custom prompts though.

Conversation mode only supports spoken input, though you can edit flawed transcriptions manually. Querying GPT-3 with nonsensical or ungrammatical text will negatively affect response quality.

Task Mode

In task mode, you can ask GPT-3 to perform a number pre-defined tasks. Written and spoken input are both supported. By default, GPT-3's response is both typed out and read aloud.

Transcripts of responses from a small subset of non-conversation tasks can be found in the data/completions directory. You can also save your own completions while using the app.

Usage Notes

The first time you speak, the speech transcription back end will take a few seconds to calibrate to the level of ambient noise in your environment. You will know it's ready for transcription when you see a "Listening..." message appear below the Record button. Calibration only occurs once to save time.

Hotkeys

CTRL + SHIFT: Start recording audio (same as pressing the "Record" button).
CTRL + a: Get GPT-3's response to whatever input you've recorded (same as pressing the "Get Response" button).

Project Members

  • Harrison Mamin

Repo Structure

jabberwocky/
β”œβ”€β”€ data         # Raw and processed data. Some files are excluded from github but the ones needed to run the app are there.
β”œβ”€β”€ notes        # Miscellaneous notes from the development process stored as raw text files.
β”œβ”€β”€ notebooks    # Jupyter notebooks for experimentation and exploratory analysis.
β”œβ”€β”€ reports      # Markdown reports (performance reports, blog posts, etc.)
β”œβ”€β”€ gui          # GUI scripts. The main script should be run from the project root directory. 
└── lib          # Python package. Code can be imported in analysis notebooks, py scripts, etc.

The docker and setup dirs contain remnants from previous attempts to package the app. While I ultimately decided to go with a simpler approach, I left them in the repo so I have the option of picking up where I left off if I decide to work on a new version.

Owner
Data Scientist
Code for ACL 2020 paper "Rigid Formats Controlled Text Generation"

SongNet SongNet: SongCi + Song (Lyrics) + Sonnet + etc. @inproceedings{li-etal-2020-rigid, title = "Rigid Formats Controlled Text Generation",

Piji Li 212 Dec 17, 2022
nlp-tutorial is a tutorial for who is studying NLP(Natural Language Processing) using Pytorch

nlp-tutorial is a tutorial for who is studying NLP(Natural Language Processing) using Pytorch. Most of the models in NLP were implemented with less than 100 lines of code.(except comments or blank li

Tae-Hwan Jung 11.9k Jan 08, 2023
Code for the paper "A Simple but Tough-to-Beat Baseline for Sentence Embeddings".

Code for the paper "A Simple but Tough-to-Beat Baseline for Sentence Embeddings".

1.1k Dec 27, 2022
Twitter bot that uses NLP models to summarize news articles referenced in a user's twitter timeline

Twitter-News-Summarizer Twitter bot that uses NLP models to summarize news articles referenced in a user's twitter timeline 1.) Extracts all tweets fr

Rohit Govindan 1 Jan 27, 2022
A calibre plugin that generates Word Wise and X-Ray files then sends them to Kindle. Supports KFX, AZW3 and MOBI eBooks. X-Ray supports 18 languages.

WordDumb A calibre plugin that generates Word Wise and X-Ray files then sends them to Kindle. Supports KFX, AZW3 and MOBI eBooks. Languages X-Ray supp

172 Dec 29, 2022
The official repository of the ISBI 2022 KNIGHT Challenge

KNIGHT The official repository holding the data for the ISBI 2022 KNIGHT Challenge About The KNIGHT Challenge asks teams to develop models to classify

Nicholas Heller 4 Jan 22, 2022
Chinese Grammatical Error Diagnosis

nlp-CGED Chinese Grammatical Error Diagnosis 中文语法纠错研穢 εŸΊδΊŽεΊεˆ—ζ ‡ζ³¨ηš„ζ–Ήζ³• ζ‰€ιœ€ηŽ―ε’ƒ Python==3.6 tensorflow==1.14.0 keras==2.3.1 bert4keras==0.10.6 η¬”θ€…δ½Ώη”¨δΊ†εΌ€ζΊηš„bert4keras

12 Nov 25, 2022
A python package for deep multilingual punctuation prediction.

This python library predicts the punctuation of English, Italian, French and German texts. We developed it to restore the punctuation of transcribed spoken language.

Oliver Guhr 27 Dec 22, 2022
Baseline code for Korean open domain question answering(ODQA)

Open-Domain Question Answering(ODQA)λŠ” λ‹€μ–‘ν•œ μ£Όμ œμ— λŒ€ν•œ λ¬Έμ„œ μ§‘ν•©μœΌλ‘œλΆ€ν„° μžμ—°μ–΄ μ§ˆμ˜μ— λŒ€ν•œ 닡변을 μ°Ύμ•„μ˜€λŠ” taskμž…λ‹ˆλ‹€. μ΄λ•Œ μ‚¬μš©μž μ§ˆμ˜μ— λ‹΅λ³€ν•˜κΈ° μœ„ν•΄ μ£Όμ–΄μ§€λŠ” 지문이 λ”°λ‘œ μ‘΄μž¬ν•˜μ§€ μ•ŠμŠ΅λ‹ˆλ‹€. λ”°λΌμ„œ 사전에 κ΅¬μΆ•λ˜μ–΄μžˆλŠ” Knowl

VUMBLEB 69 Nov 04, 2022
Rank-One Model Editing for Locating and Editing Factual Knowledge in GPT

Rank-One Model Editing (ROME) This repository provides an implementation of Rank-One Model Editing (ROME) on auto-regressive transformers (GPU-only).

Kevin Meng 130 Dec 21, 2022
Question answering app is used to answer for a user given question from user given text.

Question answering app is used to answer for a user given question from user given text.It is created using HuggingFace's transformer pipeline and streamlit python packages.

Siva Prakash 3 Apr 05, 2022
Python wrapper for Stanford CoreNLP tools v3.4.1

Python interface to Stanford Core NLP tools v3.4.1 This is a Python wrapper for Stanford University's NLP group's Java-based CoreNLP tools. It can eit

Dustin Smith 610 Sep 07, 2022
End-to-End Speech Processing Toolkit

ESPnet: end-to-end speech processing toolkit system/pytorch ver. 1.0.1 1.1.0 1.2.0 1.3.1 1.4.0 1.5.1 1.6.0 1.7.1 1.8.1 ubuntu18/python3.8/pip ubuntu18

ESPnet 5.9k Jan 03, 2023
Guide: Finetune GPT2-XL (1.5 Billion Parameters) and GPT-NEO (2.7 B) on a single 16 GB VRAM V100 Google Cloud instance with Huggingface Transformers using DeepSpeed

Guide: Finetune GPT2-XL (1.5 Billion Parameters) and GPT-NEO (2.7 Billion Parameters) on a single 16 GB VRAM V100 Google Cloud instance with Huggingfa

289 Jan 06, 2023
KakaoBrain KoGPT (Korean Generative Pre-trained Transformer)

KoGPT KoGPT (Korean Generative Pre-trained Transformer) https://github.com/kakaobrain/kogpt https://huggingface.co/kakaobrain/kogpt Model Descriptions

Kakao Brain 797 Dec 26, 2022
AI-powered literature discovery and review engine for medical/scientific papers

AI-powered literature discovery and review engine for medical/scientific papers paperai is an AI-powered literature discovery and review engine for me

NeuML 819 Dec 30, 2022
HAN2HAN : Hangul Font Generation

HAN2HAN : Hangul Font Generation

Changwoo Lee 36 Dec 28, 2022
VoilΓ  turns Jupyter notebooks into standalone web applications

Rendering of live Jupyter notebooks with interactive widgets. Introduction VoilΓ  turns Jupyter notebooks into standalone web applications. Unlike the

VoilΓ  Dashboards 4.5k Jan 03, 2023