Network automation lab using nornir, scrapli, and containerlab with Arista EOS

Related tags

Testingnornirscrapliarista-automationcontainerlabnornir-scraplinornir-jinja2
Overview

nornir-scrapli-eos-lab

Network automation lab using nornir, scrapli, and containerlab with Arista EOS.

Objectives

  1. Deploy base configs to 4xArista devices via scrapli
  2. Deploy interface configs
  3. Deploy underlay BGP configs to
  4. Deploy overlay BGP EVPN configs
  5. Provision vxlan across leaf nodes with user-defined parameters

Tools

  1. Containerlab (https://containerlab.srlinux.dev/)
  2. nornir (https://github.com/nornir-automation/nornir)
  3. nornir_utils (https://github.com/nornir-automation/nornir_utils)
  4. nornir-jinja2 (https://github.com/nornir-automation/nornir_jinja2)
  5. nornir-scrapli (https://github.com/scrapli/nornir_scrapli)

My environment

  • Ubuntu Server 20.04 LTS x86_64 with 32GB RAM running in VirtualBox (Windows 10 Pro). This lab only consumes a bit less than 4GB of RAM.
  • Containerlab version 0.16.2 (any later version should do)
  • Docker 20.10.8

Containerlab

Containerlab provides a CLI for orchestrating and managing container-based networking labs. Follow the instruction in the containerlab website to install it for Linux. https://containerlab.srlinux.dev/install/

Arista ceos images

Arista is kind enough to provide docker images of their EOS routing platform for learning and testing purpose via their website together with the instruction to get it to work with Docker. I'm using ceosimage:4.26.1F

Setup

Clone the repo:

git clone https://github.com/vireakouk/nornir-scrapli-eos-lab.git

Create a Python3 virtual environment in the project directory:

python3 -m venv env

Activate virtual environment:

source env/bin/activate

and install the following:

pip install nornir
pip install nornir-scrapli
pip install nornir_utils
pip install nornir-jinja2

or use requirements.txt

pip install -r requirements.txt

Spinning up the lab

cd containerlab/
sudo containerlab deploy -t eos.clab.yml

(use "sudo containerlab deploy -t eos.clab.yml --reconfigure" to do a clean boot if you run it not for the first time.)

Wait for a minute or two for all devices to boot up and make sure you can ssh to each device using admin/admin credential. Scrapli in this case use ssh port 22 as transport.

Deploy base config (deploy_baseconfig.py)

cd nornir-scrapli-eos-lab/
python deploy_baseconfig.py

This simple script achieves the following:

  1. create a user account,
  2. enable ip routing,
  3. config loopback0 interface with ipv4 and ipv6,

It does those by:

  1. Generate a temporary config per device by filling the template file templates/base.j2 with the hosts variables in the inventory files (hosts.yml, groups.yml, and defaults.yml)
r = task.run(task=template_file, 
                template="base.j2", 
                path="./templates")
  1. Store the resultant config to each specific host variable dictionary with an abitary key "config" :
task.host["config"] = r.result
  1. Send the config to device using scrapli send_config method:
task.run(task=send_config,
            name="Deploy base configuration on the device.",
            dry_run=False,
            config=task.host["config"])
  1. Display the new changed config using scrapli send_command method:
task.run(task=send_commands, 
            name="Show new config and copy running config to startup config.", 
            commands=["show run", "write memory"])

Deploy interface configs (deploy_interfaces.py)

cd nornir-scrapli-eos-lab/
python deploy_interfaces.py

This simple script achieves the following:

  1. config interfaces with description, ipv4, ipv6, disable switchport
  2. enable the interface with "no shutdown"

It does those by:

  1. Generate a temporary config per device by filling the template file templates/interfaces.j2 by looping through interface values in hosts variables in the inventory files (hosts.yml)
r = task.run(task=template_file, 
                template="interfaces.j2", 
                path="./templates")
  1. Store the resultant config to each specific host variable dictionary with an abitary key "config" :
task.host["config"] = r.result
  1. Send the config to device using scrapli send_config method:
task.run(task=send_config, 
            name="Configuring interfaces!", 
            dry_run=False,
            config=task.host["config"])
  1. Display the new changed config using scrapli send_command method:
task.run(task=send_commands, 
            name="Show new config and copy running config to startup config.", 
            commands=["show run", "write memory"])

Verify results of interface configs

After the deployment, if there is no error, you should be able to ping adjacent interfaces. If you can't, check the nornir.log for clues on the errors.

Deploy underlay BGP configs (deploy_underlay.py)

Follow the same concepts as the two previous tasks but using "templates/underlay.j2" instead.

Verify results of underlay configs

After the deployment, if there is no error, you should see all bgp adjacencies established.

spine1#show ip bgp summary
BGP summary information for VRF default
Router identifier 1.1.1.1, local AS number 65000
Neighbor Status Codes: m - Under maintenance
  Neighbor         V  AS           MsgRcvd   MsgSent  InQ OutQ  Up/Down State   PfxRcd PfxAcc
  10.1.1.1         4 65001              9         8    0    0 00:02:36 Estab   2      2
  10.1.1.3         4 65002              8         9    0    0 00:02:36 Estab   2      2
spine1#show ipv6 bgp summary
BGP summary information for VRF default
Router identifier 1.1.1.1, local AS number 65000
Neighbor Status Codes: m - Under maintenance
  Neighbor         V  AS           MsgRcvd   MsgSent  InQ OutQ  Up/Down State   PfxRcd PfxAcc
  2010:1:1:2::1    4 65001             22        22    0    0 00:11:37 Estab   3      3
  2010:1:1:2::3    4 65002             20        19    0    0 00:11:38 Estab   3      3
spine1#

spine2#show ip bgp sum
BGP summary information for VRF default
Router identifier 1.1.1.2, local AS number 65000
Neighbor Status Codes: m - Under maintenance
  Neighbor         V  AS           MsgRcvd   MsgSent  InQ OutQ  Up/Down State   PfxRcd PfxAcc
  10.1.1.5         4 65001             18        17    0    0 00:10:19 Estab   3      3
  10.1.1.7         4 65002             19        20    0    0 00:10:19 Estab   3      3
spine2#show ipv6 bgp sum
BGP summary information for VRF default
Router identifier 1.1.1.2, local AS number 65000
Neighbor Status Codes: m - Under maintenance
  Neighbor         V  AS           MsgRcvd   MsgSent  InQ OutQ  Up/Down State   PfxRcd PfxAcc
  2010:1:1:2::5    4 65001             19        21    0    0 00:12:04 Estab   2      2
  2010:1:1:2::7    4 65002             19        19    0    0 00:12:04 Estab   2      2
spine2#

leaf1#show ip bgp summary
BGP summary information for VRF default
Router identifier 1.1.1.3, local AS number 65001
Neighbor Status Codes: m - Under maintenance
  Neighbor         V  AS           MsgRcvd   MsgSent  InQ OutQ  Up/Down State   PfxRcd PfxAcc
  10.1.1.0         4 65000             18        18    0    0 00:10:46 Estab   2      2
  10.1.1.4         4 65000             18        19    0    0 00:10:45 Estab   2      2
leaf1#show ipv6 bgp sum
BGP summary information for VRF default
Router identifier 1.1.1.3, local AS number 65001
Neighbor Status Codes: m - Under maintenance
  Neighbor         V  AS           MsgRcvd   MsgSent  InQ OutQ  Up/Down State   PfxRcd PfxAcc
  2010:1:1:2::     4 65000             22        23    0    0 00:12:22 Estab   3      3
  2010:1:1:2::4    4 65000             21        20    0    0 00:12:22 Estab   3      3
leaf1#

leaf2#show ip bgp summary
BGP summary information for VRF default
Router identifier 1.1.1.4, local AS number 65002
Neighbor Status Codes: m - Under maintenance
  Neighbor         V  AS           MsgRcvd   MsgSent  InQ OutQ  Up/Down State   PfxRcd PfxAcc
  10.1.1.2         4 65000             19        18    0    0 00:11:13 Estab   3      3
  10.1.1.6         4 65000             21        20    0    0 00:11:13 Estab   3      3
leaf2#show ipv6 bgp summary
BGP summary information for VRF default
Router identifier 1.1.1.4, local AS number 65002
Neighbor Status Codes: m - Under maintenance
  Neighbor         V  AS           MsgRcvd   MsgSent  InQ OutQ  Up/Down State   PfxRcd PfxAcc
  2010:1:1:2::2    4 65000             20        22    0    0 00:12:44 Estab   2      2
  2010:1:1:2::6    4 65000             20        20    0    0 00:12:43 Estab   2      2
leaf2#

leaf1#show ip bgp
BGP routing table information for VRF default
Router identifier 1.1.1.3, local AS number 65001
Route status codes: s - suppressed, * - valid, > - active, E - ECMP head, e - ECMP
                    S - Stale, c - Contributing to ECMP, b - backup, L - labeled-unicast
                    % - Pending BGP convergence
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI Origin Validation codes: V - valid, I - invalid, U - unknown
AS Path Attributes: Or-ID - Originator ID, C-LST - Cluster List, LL Nexthop - Link Local Nexthop

          Network                Next Hop              Metric  AIGP       LocPref Weight  Path
 * >      1.1.1.1/32             10.1.1.0              0       -          100     0       65000 i
 * >      1.1.1.2/32             10.1.1.4              0       -          100     0       65000 i
 * >      1.1.1.3/32             -                     -       -          -       0       i
 * >Ec    1.1.1.4/32             10.1.1.0              0       -          100     0       65000 65002 i
 *  ec    1.1.1.4/32             10.1.1.4              0       -          100     0       65000 65002 i
leaf1#

Deploy overlay BGP configs (deploy_overlay.py)

Follow the same concepts as the two previous tasks but using "templates/overlay.j2" instead.

Verify results of overlay configs

spine1#show bgp evpn summary
BGP summary information for VRF default
Router identifier 1.1.1.1, local AS number 65000
Neighbor Status Codes: m - Under maintenance
  Neighbor         V  AS           MsgRcvd   MsgSent  InQ OutQ  Up/Down State   PfxRcd PfxAcc
  1.1.1.3          4 65001             23        24    0    0 00:16:51 Estab   0      0
  1.1.1.4          4 65002             23        24    0    0 00:16:51 Estab   0      0
  2001:1:1:1::3    4 65001             23        23    0    0 00:16:51 Estab   0      0
  2001:1:1:1::4    4 65002             23        23    0    0 00:16:51 Estab   0      0
spine1#

spine2#show bgp evpn summary
BGP summary information for VRF default
Router identifier 1.1.1.2, local AS number 65000
Neighbor Status Codes: m - Under maintenance
  Neighbor         V  AS           MsgRcvd   MsgSent  InQ OutQ  Up/Down State   PfxRcd PfxAcc
  1.1.1.3          4 65001             25        25    0    0 00:17:24 Estab   0      0
  1.1.1.4          4 65002             24        24    0    0 00:17:24 Estab   0      0
  2001:1:1:1::3    4 65001             23        24    0    0 00:17:24 Estab   0      0
  2001:1:1:1::4    4 65002             24        26    0    0 00:17:24 Estab   0      0
spine2#

leaf1#show bgp evpn summary
BGP summary information for VRF default
Router identifier 1.1.1.3, local AS number 65001
Neighbor Status Codes: m - Under maintenance
  Neighbor         V  AS           MsgRcvd   MsgSent  InQ OutQ  Up/Down State   PfxRcd PfxAcc
  1.1.1.1          4 65000             22        21    0    0 00:15:13 Estab   0      0
  1.1.1.2          4 65000             22        22    0    0 00:15:13 Estab   0      0
  2001:1:1:1::1    4 65000             21        21    0    0 00:15:13 Estab   0      0
  2001:1:1:1::2    4 65000             22        21    0    0 00:15:13 Estab   0      0
leaf1#

leaf2#show bgp evpn summary
BGP summary information for VRF default
Router identifier 1.1.1.4, local AS number 65002
Neighbor Status Codes: m - Under maintenance
  Neighbor         V  AS           MsgRcvd   MsgSent  InQ OutQ  Up/Down State   PfxRcd PfxAcc
  1.1.1.1          4 65000             25        25    0    0 00:18:04 Estab   0      0
  1.1.1.2          4 65000             24        25    0    0 00:18:04 Estab   0      0
  2001:1:1:1::1    4 65000             25        25    0    0 00:18:03 Estab   0      0
  2001:1:1:1::2    4 65000             26        26    0    0 00:18:04 Estab   0      0
leaf2#

Provision Vxlan (provision_l2vxlan.py)

The script takes user inputs such as vlan ID, VNI, name, A-end device, and B-end. It stores the user input in the host dictionary with abitrary key ["vxlan"] and deploy the configs to devices of user's inputs in A-end and B-end.

from nornir import InitNornir
from nornir.core.task import Task, Result
from nornir.core.filter import F
from nornir_utils.plugins.functions import print_result
from nornir_jinja2.plugins.tasks import template_file
from nornir_scrapli.tasks import send_commands, send_config

def deploy_l2vxlan(task: Task, data: dict) -> Result:
    task.host["vxlan"] = data
    r = task.run(task=template_file, 
                template="l2vxlan.j2",
                path="./templates")
    task.host["config"] = r.result


    task.run(task=send_config, 
            name="Provisioning L2 VXLAN!", 
            dry_run=False,
            config=task.host["config"])
    
    task.run(task=send_commands, 
            name="Show new config and copy running config to startup config.", 
            commands=["show run", "write memory"])

def get_input() -> dict:
    print("*"*62)
    print("* This script will provision L2 VxLAN circuit in leaf nodes. *")
    print("*"*62)
    data = {}
    data["vlan_id"] = input("Enter the vlan ID (ex: 10): ")
    data["vlan_name"] = input("Enter the vlan name (ex: CUST-ABC): ")
    data["vni"] = input("Enter the vni (ex: 10010): ")
    data["a_end"] = input("Enter the A-end switch name: ")
    data["b_end"] = input("Enter the B-end switch name: ")
    
    return data


if __name__ == "__main__":
    user_input = get_input()
    nr = InitNornir(config_file="config.yml")
    try:
        nr = nr.filter(F(switchname=user_input["a_end"]) | F(switchname=user_input["b_end"]))
        r = nr.run(task=deploy_l2vxlan, data=user_input)
        print_result(r)
    except KeyError as e:
        print(f"Could not find device: {e}")

~/projects/nornir-scrapli-eos-lab master* ❯ python provision_l2vxlan.py                                                                                                                 8s nornir-scrapli-eos-lab
**************************************************************
* This script will provision L2 VxLAN circuit in leaf nodes. *
**************************************************************
Enter the vlan ID (ex: 10): 11
Enter the vlan name (ex: CUST-ABC): CUST-TEST
Enter the vni (ex: 10010): 10011
Enter the A-end switch name: leaf1
Enter the B-end switch name: leaf2
/home/vireak/projects/nornir-scrapli-eos-lab/env/lib/python3.8/site-packages/scrapli/helper.py:290: UserWarning:

******************************************************************************************** Authentication Warning! *********************************************************************************************
    scrapli will try to escalate privilege without entering a password but may fail.
Set an 'auth_secondary' password if your device requires a password to increase privilege, otherwise ignore this message.
******************************************************************************************************************************************************************************************************************

  warn(warning_message)
deploy_l2vxlan******************************************************************
* leaf1.eos ** changed : True **************************************************
vvvv deploy_l2vxlan ** changed : False vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv INFO
---- template_file ** changed : False ------------------------------------------ INFO
vlan 11
 name CUST-TEST
!
interface vxlan 1
 vxlan vlan 11 vni 10011
 vxlan source-int loopback0
 vxlan udp-port 4789
 vxlan learn-restrict any
!
router bgp 65001
 !
 vlan 11
  rd 65001:10011
  route-target both 11:10011
  redistribute learned
!
---- Provisioning L2 VXLAN! ** changed : True ---------------------------------- INFO
vlan 11
 name CUST-TEST
!
interface vxlan 1
 vxlan vlan 11 vni 10011
 vxlan source-int loopback0
 vxlan udp-port 4789
 vxlan learn-restrict any
!
router bgp 65001
 !
 vlan 11
  rd 65001:10011
  route-target both 11:10011
  redistribute learned
!

---- Show new config and copy running config to startup config. ** changed : False  INFO

Verify vxlan provisioning

leaf1>show interfaces vxlan 1
Vxlan1 is up, line protocol is up (connected)
  Hardware is Vxlan
  Source interface is Loopback0 and is active with 1.1.1.3
  Replication/Flood Mode is headend with Flood List Source: EVPN
  Remote MAC learning via EVPN
  VNI mapping to VLANs
  Static VLAN to VNI mapping is
    [64, 10064]
  Note: All Dynamic VLANs used by VCS are internal VLANs.
        Use 'show vxlan vni' for details.
  Static VRF to VNI mapping is not configured
  Headend replication flood vtep list is:
    64 1.1.1.4
  MLAG Shared Router MAC is 0000.0000.0000
leaf1>show vxlan vtep
Remote VTEPS for Vxlan1:

VTEP          Tunnel Type(s)
------------- --------------
1.1.1.4       flood

Total number of remote VTEPS:  1
leaf1>

leaf2#show interfaces vxlan 1
Vxlan1 is up, line protocol is up (connected)
  Hardware is Vxlan
  Source interface is Loopback0 and is active with 1.1.1.4
  Replication/Flood Mode is headend with Flood List Source: EVPN
  Remote MAC learning via EVPN
  VNI mapping to VLANs
  Static VLAN to VNI mapping is
    [11, 10011]       [64, 10064]
  Note: All Dynamic VLANs used by VCS are internal VLANs.
        Use 'show vxlan vni' for details.
  Static VRF to VNI mapping is not configured
  Headend replication flood vtep list is:
    11 1.1.1.3
    64 1.1.1.3
  MLAG Shared Router MAC is 0000.0000.0000
leaf2#show vxlan vtep
Remote VTEPS for Vxlan1:

VTEP          Tunnel Type(s)
------------- --------------
1.1.1.3       flood

Total number of remote VTEPS:  1
leaf2#

Extension of script functionality

The template is standard jinja2 template which you can add more parameters to the base or interface configs. For example in base config, we can add further template for aaa, logging, ntp, logging..etc.

More configuration tasks

(coming soon)

Owner
Vireak Ouk
Started learning to code at the age of 35. This is my journey.
Vireak Ouk
GitHub Repository
Descriptor Vector Exchange

Descriptor Vector Exchange This repo provides code for learning dense landmarks without supervision. Our approach is described in the ICCV 2019 paper

James Thewlis 74 Nov 29, 2022
Automatic SQL injection and database takeover tool

sqlmap sqlmap is an open source penetration testing tool that automates the process of detecting and exploiting SQL injection flaws and taking over of

sqlmapproject 25.7k Jan 04, 2023
An interactive TLS-capable intercepting HTTP proxy for penetration testers and software developers.

mitmproxy mitmproxy is an interactive, SSL/TLS-capable intercepting proxy with a console interface for HTTP/1, HTTP/2, and WebSockets. mitmdump is the

mitmproxy 29.7k Jan 02, 2023
Getting the most out of your hobby servo

ServoProject by Adam Bäckström Getting the most out of your hobby servo Theory The control system of a regular hobby servo looks something like this:

209 Dec 20, 2022
Python selenium script to bypass simaster.ugm.ac.id weak captcha.

Python selenium script to bypass simaster.ugm.ac.id weak "captcha".

Hafidh R K 1 Feb 01, 2022
Automating the process of sorting files in my downloads folder by file type.

downloads-folder-automation Automating the process of sorting files in a user's downloads folder on Windows by file type. This script iterates through

Eric Mahasi 27 Jan 07, 2023
create custom test databases that are populated with fake data

About Generate fake but valid data filled databases for test purposes using most popular patterns(AFAIK). Current support is sqlite, mysql, postgresql

Emir Ozer 2.2k Jan 04, 2023
A pytest plugin that enables you to test your code that relies on a running Elasticsearch search engine

pytest-elasticsearch What is this? This is a pytest plugin that enables you to test your code that relies on a running Elasticsearch search engine. It

Clearcode 65 Nov 10, 2022
Make Selenium work on Github Actions

Make Selenium work on Github Actions Scraping with BeautifulSoup on GitHub Actions is easy-peasy. But what about Selenium?? After you jump through som

Jonathan Soma 33 Dec 27, 2022
Nokia SR OS automation

Nokia SR OS automation Nokia is one of the biggest vendors of the telecommunication equipment, which is very popular in the Service Provider segment.

Karneliuk.com 7 Jul 23, 2022
A toolbar overlay for debugging Flask applications

Flask Debug-toolbar This is a port of the excellent django-debug-toolbar for Flask applications. Installation Installing is simple with pip: $ pip ins

863 Dec 29, 2022
Divide full port scan results and use it for targeted Nmap runs

Divide Et Impera And Scan (and also merge the scan results) DivideAndScan is used to efficiently automate port scanning routine by splitting it into 3

snovvcrash 226 Dec 30, 2022
A configurable set of panels that display various debug information about the current request/response.

Django Debug Toolbar The Django Debug Toolbar is a configurable set of panels that display various debug information about the current request/respons

Jazzband 7.3k Jan 02, 2023
Pytest modified env

Pytest plugin to fail a test if it leaves modified os.environ afterwards.

wemake.services 7 Sep 11, 2022
py.test fixture for benchmarking code

Overview docs tests package A pytest fixture for benchmarking code. It will group the tests into rounds that are calibrated to the chosen timer. See c

Ionel Cristian Mărieș 1k Jan 03, 2023
Generate random test credit card numbers for testing, validation and/or verification purposes.

Generate random test credit card numbers for testing, validation and/or verification purposes.

Dark Hunter 141 5 Nov 14, 2022
Automated testing tool developed in python for Advanced mathematical operations.

Advanced-Maths-Operations-Validations Automated testing tool developed in python for Advanced mathematical operations. Requirements Python 3.5 or late

Nikhil Repale 1 Nov 16, 2021
API mocking with Python.

apyr apyr (all lowercase) is a simple & easy to use mock API server. It's great for front-end development when your API is not ready, or when you are

Umut Seven 55 Nov 25, 2022
A single module to link Python ecosystem to the Web

A single module to link Python ecosystem to the Web. Have a quick look at the Gallery first to get convinced ! FAQ For any questions, please use Stack

66 Dec 21, 2022
It helps to use fixtures in pytest.mark.parametrize

pytest-lazy-fixture Use your fixtures in @pytest.mark.parametrize. Installation pip install pytest-lazy-fixture Usage import pytest @pytest.fixture(p

Marsel Zaripov 299 Dec 24, 2022