A lobby boy will create a VPS server when you need one, and destroy it after using it.

Overview

Lobbyboy

What is a lobby boy? A lobby boy is completely invisible, yet always in sight. A lobby boy remembers what people hate. A lobby boy anticipates the client's needs before the needs are needed. A lobby boy is, above all, discreet to a fault.

--The Grand Budapest Hotel

This project is still under testing, it worked but may have bugs.

What is lobbyboy?

Well, lobbyboy is a ssh server. Yes, like sshd. But instead of spawn a new shell on the server like sshd, when you ssh to lobbyboy, lobbyboy will create a new server(VPS) from available providers(meaning to say, DigitalOcean, AWS, GCP, Vultr, etc), then redirect you to the newly created servers. Of course, if lobbyboy finds any servers available already, he will just ask if you want to enter the existing server, or still want to create a new one.

                                                       create
 +------------------+          +--------------------+  new server  +--------------------------+
 |                  |          |                    |------------->|                          |
 User(You!)         |--ssh----->    lobbyboy        |              |  DigitalOcean            |
 |                  |          |                    |------------->|  (or any other providers |
 +------------------+          +--------------------+     ssh      +--------------------------+

Key Features

  • talks in SSH2 protocol, no need to install any software of configs for client-side, just ssh to lobbyboy!
  • extendable provider: just implement 3 methods, then lobbyboy can work with any provider!
  • destroy the server when you no longer needed.
  • manage ssh keys for you

Installation

Install via pip:

pip install lobbyboy

Then generate config file:

lobbyboy-config-example > config.toml
# Edit your config before running!

Run server

lobbyboy-server -c config.toml

Providers

// TBD

Builtin Providers

Lobbyboy current support two Providers:

  • DigitalOcean
  • Vagrant (Need vagrant and virtualbox to be installed)

Different Providers support different configs, please see the example config for more detail.

Write Your Own Providers

Providers are VPS vendors, by writing new providers, lobbyboy can work with any VPS vendors.

To make a new Provider work, you need to extend base class `lobbyboy.provider.BaseProvider``, implement 3 methods:

  def new_server(self, channel):
      """
      Args:
          channel: paramiko channel

      Returns:
          created_server_id: unique id from provision
          created_server_host: server's ip or domain address
      """
      pass

  def destroy_server(self, server_id, server_ip, channel):
      """
      Args:
          channel: Note that the channel can be None.
                    If called from server_killer, channel will be None.
                    if called when user logout from server, channel is active.
      """
      pass

  def ssh_server_command(self, server_id, server_ip):
      """
      Args:
          server_id: the server ssh to, which is returned by you from ``new_server``
          server_ip: ip or domain name.
      Returns:
          list: a command in list format, for later to run exec.
      """
      pass

Then add your Provider to your config file.

Those 3 configs are obligatory, as lobbyboy has to know when should he destroy your spare servers. You can add more configs, and read them from self.provider_config from code, just remember to add docs about it :)

[provider.<your provider name>]
loadmodule = "lobbyboy.contrib.provider.vagrant::VagrantProvider"
min_life_to_live = "1h"
bill_time_unit = "1h"

FAQ

Q: Can I use lobbyboy as a proxy, like adding it to my ProxyCommand in ssh config?

A: No. Lobbyboy works like a reverse proxy, meaning to say, for ssh client, it just like a ssh server(sshd maybe), ssh client get a shell from lobbyboy, and doesn't know if it is local shell or it is a nested shell which runs another ssh. (but you know it, right? :D )

Comments
  • LBConfigProvider is inflexible

    LBConfigProvider is inflexible

    My original idea for the provider is that:

    A provider can be defined in a single file, a provider's exceptions, implementations, and config definitions, all can be defined in a single file.

    So that it would be easy to maintain. And also, a provider can be implemented outside lobbyboy's codebase. (let's say, one can upload his provider to pypi, and other users can install them via pip install lobbyboy-aws-ec2-provider, then lobbyboy can load it.

    And one can add custom configs to his provider.

    The current problem is, we have to update the LBConfigProvider if one wants to add more fields to his provider config. It is not possible to do so without updating lobbyboy's source code.

    opened by laixintao 7
  • UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 4093-4094: unexpected end of data

    UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 4093-4094: unexpected end of data

    CRI [20211208-07:42:34.995] thr=140672309044992 lobbyboy.socket_handle:254: *** Socket thread error.                                                                                                                                                                                                                          Traceback (most recent call last):
      File "/root/code/lobbyboy/lobbyboy/socket_handle.py", line 247, in run
        self.user_using(server, proxy_subprocess)
      File "/root/code/lobbyboy/lobbyboy/socket_handle.py", line 188, in user_using
        send_to_channel(self.channel, os.read(master_fd, 10240).decode(), suffix="")
    UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 4093-4094: unexpected end of data
    DEB [20211208-07:42:35.003] thr=140672300570368 paramiko.transport:1819: EOF in transport thread
    
    opened by messense 5
  • Initialization Problems.

    Initialization Problems.

    I am really interested in your idea of lobbyboy. But I know nothing about python.

    So here is my problems during initialization, can you help me to figure it out?

    1. I configure my own data_dir but service start with an error. obbyboy.utils: Error when reading available_servers.json, [Errno 2] No such file or directory: '~/Software/lobbyboy/data/available_servers.json' Even I create the available_servers.json file, it does not work. Until I write {} to the file. But I'm not sure if this is ok for the lobbyboy service
    2. When I ssh with ssh [email protected] -p 12200, the logs shows an error below:
    ERR [20211119-17:40:41.664] thr=123145413386240 paramiko.transport: Unknown exception: too many values to unpack (expected 2)
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport: Traceback (most recent call last):
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport:   File "/usr/local/lib/python3.9/site-packages/paramiko/transport.py", line 2109, in run
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport:     handler(self.auth_handler, m)
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport:   File "/usr/local/lib/python3.9/site-packages/paramiko/auth_handler.py", line 525, in _parse_userauth_request
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport:     result = self.transport.server_object.check_auth_publickey(
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport:   File "/usr/local/lib/python3.9/site-packages/lobbyboy/server.py", line 103, in check_auth_publickey
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport:     _, _key_pub = line.split(" ", 2)
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport: ValueError: too many values to unpack (expected 2)
    

    I did some research but it's really difficult for me to understand python.

    So if you can give me a hand, that's would be nice.

    Thank you.

    opened by kimichen13 4
  • Use pre-commit as the linter

    Use pre-commit as the linter

    Close #48

    Install the commit hooks in dev:

    poetry run pre-commit install
    

    Run in CI: I recommend using pre-commit ci, it can upgrade the hooks and commit linter changes automatically. It is also blazing fast, e2e run time is < 10s

    BTW, the code is changed by the linter so this is better to merge after #55, otherwise there will be lots of conflicts

    opened by frostming 3
  • Recommended use pre-commit to make code style consistency

    Recommended use pre-commit to make code style consistency

    @laixintao @messense @frostming

    Recommended use pre-commit to make code style consistency, especially in the case of multi-person cooperation, what do you think?

    opened by luxiaba 3
  • Bump paramiko from 2.8.1 to 2.10.1

    Bump paramiko from 2.8.1 to 2.10.1

    Bumps paramiko from 2.8.1 to 2.10.1.

    Commits
    • 286bd9f Cut 2.10.1
    • 4c491e2 Fix CVE re: PKey.write_private_key chmod race
    • aa3cc6f Cut 2.10.0
    • e50e19f Fix up changelog entry with real links
    • 02ad67e Helps to actually leverage your mocked system calls
    • 29d7bf4 Clearly our agent stuff is not fully tested yet...
    • 5fcb8da OpenSSH docs state %C should also work in IdentityFile and Match exec
    • 1bf3dce Changelog enhancement
    • f6342fc Prettify, add %C as acceptable controlpath token, mock gethostname
    • 3f3451f Add to changelog
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • OSError: [Errno 9] Bad file descriptor

    OSError: [Errno 9] Bad file descriptor

    INF [20211208-07:44:51.056] thr=140672309044992 lobbyboy.utils:127: user choose 1 for option There are 1 available servers:
    INF [20211208-07:44:51.056] thr=140672309044992 lobbyboy.socket_handle:79: user choose server input=1.
    DEB [20211208-07:44:51.057] thr=140672309044992 lobbyboy.contrib.provider.ignite:86: get ssh to server command for ignite: ['cd dev_datadir/ignite/2021-12-08-0739 && ignite ssh 2021-12-08-0739']
    INF [20211208-07:44:51.057] thr=140672309044992 lobbyboy.socket_handle:103: ssh to server 2021-12-08-0739 127.0.0.1: cd dev_datadir/ignite/2021-12-08-0739 && ignite ssh 2021-12-08-0739
    INF [20211208-07:44:51.062] thr=140672309044992 lobbyboy.socket_handle:176: proxy subprocess created, pid=71892
    CRI [20211208-07:44:51.987] thr=140672309044992 lobbyboy.socket_handle:254: *** Socket thread error.
    Traceback (most recent call last):
      File "/root/code/lobbyboy/lobbyboy/socket_handle.py", line 252, in run
        self.cleanup(t, meta=lb_server, check_destroy=True)
      File "/root/code/lobbyboy/lobbyboy/socket_handle.py", line 194, in cleanup
        self.remove_server_session(t, meta.server_name)
      File "/root/code/lobbyboy/lobbyboy/socket_handle.py", line 223, in remove_server_session
        active_session[server_name] = list(filter(lambda x: x.getpeername() != peer_name, sessions))
      File "/root/code/lobbyboy/lobbyboy/socket_handle.py", line 223, in <lambda>
        active_session[server_name] = list(filter(lambda x: x.getpeername() != peer_name, sessions))
      File "/root/.cache/pypoetry/virtualenvs/lobbyboy-P-fHd6h2-py3.8/lib/python3.8/site-packages/paramiko/transport.py", line 1787, in getpeername
        return gp()
    OSError: [Errno 9] Bad file descriptor
    
    opened by messense 1
  • tests for need_destroy

    tests for need_destroy

    somehow some of my instance didn't get destroyed, I suspect there are some bugs in https://github.com/lobbyboy-ssh/lobbyboy/blob/41bba07e856e6ce31f7164232723e7440a469f18/lobbyboy/server_killer.py#L42

    but I didn't find any, let me add test cases later.

    opened by laixintao 3
  • Security check when lobbyboy server starts.

    Security check when lobbyboy server starts.

    Need to check:

    • user started lobbyboy with default username/password?
    • with the ssh host key https://github.com/lobbyboy-ssh/lobbyboy/blob/main/dev_datadir/ssh_host_rsa_key in git repo (middle man attack)

    If yes, then need to print warning information to stderr.

    related discussion: https://github.com/lobbyboy-ssh/lobbyboy/pull/32

    opened by laixintao 0
  • vultr provider support

    vultr provider support

    i want to add vultr provider and i found python-vultr, but seems that it hasn't been updated for a long time, and there are fatal bugs that can't be used normally, i write a new one pyvultr, can we use this to support vultr provider?

    opened by luxiaba 2
Releases(v0.4.0)
  • v0.4.0(Dec 11, 2021)

    What's Changed

    • fix typo of exmaple config file name. by @laixintao in https://github.com/lobbyboy-ssh/lobbyboy/pull/25
    • Fix typos and add codespell check to CI by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/26
    • Fix the path of example config.toml by @frostming in https://github.com/lobbyboy-ssh/lobbyboy/pull/28
    • Support docker container (based on footloose) by @laixintao in https://github.com/lobbyboy-ssh/lobbyboy/pull/30
    • ci: fix unittest running branch master-> main by @laixintao in https://github.com/lobbyboy-ssh/lobbyboy/pull/36
    • Add instructions to run in docker by @frostming in https://github.com/lobbyboy-ssh/lobbyboy/pull/32
    • Add ignite provider by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/37
    • Revamp provider config by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/41
    • Fix send_to_channel by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/44
    • Add support for enable/disable provider in config by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/45
    • Add a test case for load_config by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/49
    • Add a test case for load_providers by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/50
    • Add coverage report to CI by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/51
    • update readme: new architecture picture, update new providers. by @laixintao in https://github.com/lobbyboy-ssh/lobbyboy/pull/46
    • Define a default provider if there is only one provider by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/47

    New Contributors

    • @messense made their first contribution in https://github.com/lobbyboy-ssh/lobbyboy/pull/26
    • @frostming made their first contribution in https://github.com/lobbyboy-ssh/lobbyboy/pull/28

    Full Changelog: https://github.com/lobbyboy-ssh/lobbyboy/compare/v0.3.0...v0.4.0

    Source code(tar.gz)
    Source code(zip)
Owner
I love system administration as much as I love programming.
Micro Data Lake based on Docker Compose

Micro Data Lake based on Docker Compose This is the implementation of a Minimum Data Lake

Abel Coronado 15 Jan 07, 2023
Chartreuse: Automated Alembic migrations within kubernetes

Chartreuse: Automated Alembic SQL schema migrations within kubernetes "How to automate management of Alembic database schema migration at scale using

Wiremind 8 Oct 25, 2022
A Kubernetes operator that creates UptimeRobot monitors for your ingresses

This operator automatically creates uptime monitors at UptimeRobot for your Kubernetes Ingress resources. This allows you to easily integrate uptime monitoring of your services into your Kubernetes d

Max 49 Dec 14, 2022
Bash-based Python-venv convenience wrapper

venvrc Bash-based Python-venv convenience wrapper. Demo Install Copy venvrc file to ~/.venvrc, and add the following line to your ~/.bashrc file: # so

1 Dec 29, 2022
Ganeti is a virtual machine cluster management tool built on top of existing virtualization technologies such as Xen or KVM and other open source software.

Ganeti 3.0 =========== For installation instructions, read the INSTALL and the doc/install.rst files. For a brief introduction, read the ganeti(7) m

395 Jan 04, 2023
CDK Template of Table Definition AWS Lambda for RDB

CDK Template of Table Definition AWS Lambda for RDB Overview This sample deploys Amazon Aurora of PostgreSQL or MySQL with AWS Lambda that can define

AWS Samples 5 May 16, 2022
gunicorn 'Green Unicorn' is a WSGI HTTP Server for UNIX, fast clients and sleepy applications.

Gunicorn Gunicorn 'Green Unicorn' is a Python WSGI HTTP Server for UNIX. It's a pre-fork worker model ported from Ruby's Unicorn project. The Gunicorn

Benoit Chesneau 8.7k Jan 08, 2023
A collection of beginner-friendly DevOps content

mansion Mansion is just a testing repo for learners to commit into open source project. These are the steps you need to learn: Please do not edit thes

Bryan Lim 62 Nov 30, 2022
A lobby boy will create a VPS server when you need one, and destroy it after using it.

Lobbyboy What is a lobby boy? A lobby boy is completely invisible, yet always in sight. A lobby boy remembers what people hate. A lobby boy anticipate

226 Dec 29, 2022
HXVM - Check Host compatibility with the Virtual Machines

HXVM - Check Host compatibility with the Virtual Machines. Features | Installation | Usage Features Takes input from user to compare how many VMs they

Aman Srivastava 4 Oct 15, 2022
Get Response Of Container Deployment Kube with python

get-response-of-container-deployment-kube 概要 get-response-of-container-deployment-kube は、例えばエッジコンピューティング環境のコンテナデプロイメントシステムにおいて、デプロイ元の端末がデプロイ先のコンテナデプロイ

Latona, Inc. 3 Nov 05, 2021
Tencent Yun tools with python

Tencent_Yun_tools 使用 python3.9 + 腾讯云 AccessKey 利用工具 使用之前请先填写config.ini配置文件 Usage python3 Tencent_rce.py -h Scanner python3 Tencent_rce.py -s 生成CSV

<img src="> 13 Dec 20, 2022
A Python Implementation for Git for learning

A pure Python implementation for Git based on Buliding Git

shidenggui 42 Jul 13, 2022
DC/OS - The Datacenter Operating System

DC/OS - The Datacenter Operating System The easiest way to run microservices, big data, and containers in production. What is DC/OS? Like traditional

DC/OS 2.3k Jan 06, 2023
MLops tools review for execution on multiple cluster types: slurm, kubernetes, dask...

MLops tools review focused on execution using multiple cluster types: slurm, kubernetes, dask...

4 Nov 30, 2022
This project shows how to serve an TF based image classification model as a web service with TFServing, Docker, and Kubernetes(GKE).

Deploying ML models with CPU based TFServing, Docker, and Kubernetes By: Chansung Park and Sayak Paul This project shows how to serve a TensorFlow ima

Chansung Park 104 Dec 28, 2022
Spinnaker is an open source, multi-cloud continuous delivery platform for releasing software changes with high velocity and confidence.

Welcome to the Spinnaker Project Spinnaker is an open-source continuous delivery platform for releasing software changes with high velocity and confid

8.8k Jan 07, 2023
A simple python application for running a CI pipeline locally This app currently supports GitLab CI scripts

🏃 Simple Local CI Runner 🏃 A simple python application for running a CI pipeline locally This app currently supports GitLab CI scripts ⚙️ Setup Inst

Tom Stowe 0 Jan 11, 2022
A charmed operator for running PGbouncer on kubernetes.

operator-template Description TODO: Describe your charm in a few paragraphs of Markdown Usage TODO: Provide high-level usage, such as required config

Canonical 1 Dec 01, 2022
Find-Xss - Termux Kurulum Dosyası Eklendi Eğer Hata Alıyorsanız Lütfen Resmini Çekip İnstagramdan Bildiriniz

FindXss Waf Bypass Eklendi !!! PRODUCER: Saep UPDATER: Aser-Vant Download: git c

Aser 2 Apr 17, 2022