TLDR: pyATS is an automation screening framework that features a parsing library known as Genie. With around 1500 parsers out there, Genie can parse gadget output from numerous suppliers, like Cisco, Juniper, and Major-IP. In combination with pyATS, you have a finish examination suite that can offer confidence your network is functioning nutritious.

Have you ever been asked by your supervisor, “Can we affirm feature ‘X’ is configured and operating across every gadget in our network?” This may perhaps be a uncomplicated characteristic this sort of as SNMPv3, or a little something more sophisticated like a distinct routing layout. Validating the operation of your community at any position in time can be a tough job. Having said that, we can ease that suffering with an automation screening framework, these types of as pyATS.

Cisco pyATS is an automation screening framework that Cisco has designed and used internally to exam/validate features in their NOS platforms. It was produced to the general public in 2014. The core pyATS framework is even now shut-resource, as the code is not publicly readily available. However, the companion parsing library, Genie (you could also listen to it referred to as pyATS library), has been open up-sourced and encourages the public to lead. This post will aim on the pyATS framework.

pyATS OVERVIEW

The pyATS framework is expansive and can seem complicated with the selection of features it presents. The objective of this weblog put up is to keep at the “10,000 ft view”. There may perhaps be further more site posts that dive further into the particular person characteristics.

PyATS is a test automation framework, intended to create and run regular exams from your infrastructure. The companion parsing library I mentioned before, Genie, is built to be used with pyATS. Collectively, these two libraries create a complete check suite, with a testing framework and vendor-agnostic info parsers. The Genie library can be a entire separate site write-up, but the 1 takeaway is that it’s a powerful facts parsing library that couples extremely well with pyATS, but it is separate from pyATS.

pyATS Components

Now that you know that pyATS is employed for constructing and functioning assessments towards your infrastructure, let’s dive into some of the factors that make up the library.

TESTBED

The testbed is essentially your device inventory file. Other automation frameworks, these as Nornir and Ansible, have a comparable idea. A testbed file is a YAML file that describes the products you are jogging assessments against. Some of the crucial parts incorporate the machine hostname, IP address, OS form (utilised to dictate which Genie parsers to use), and a established of qualifications to join to the unit. There are loads far more knowledge points you can include in the testbed, such as how to connect to the system (CLI, YANG, Relaxation). You can even describe the screening environment’s topology by defining product interfaces and how the equipment link to a single a different by defining hyperlinks in your testbed file. If you are fascinated in understanding far more about the diverse info points, verify out the links in the references at the finish of the article.

TESTCASE

A testcase is a selection of smaller sized assessments, aiming to validate a certain element or features. For illustration, you may well generate a testcase to validate that BGP is up and operational. This testcase may well consist of more compact assessments that validate BGP neighbor interactions, BGP routes are current in the routing table, and so on. The individual check final results roll up to the testcase end result. If you’d like to discover more about testcases and other sections that make up a testscript, look at out the back links in the references at the finish of the put up.

TESTSCRIPT

A testscript is a Python file made use of to composition take a look at sections. Every single testscript has its individual reporting and logging. Testscripts are intended to be extensible, so that you can add testcases in the long run. A testscript can be executed as a standalone script, with success printed to STDOUT, or as component of a career. Standalone execution is well-liked for rapid progress but ought to be executed as aspect of a task as soon as it is prepared for creation use.

Work

Employment in pyATS enable you to operate numerous testscripts. Within just a task, every single testscript is executed as a undertaking. Each individual task aggregates its logs and final results to a single log file and reporter item. The logging and reporting mechanisms inside of a job can be a independent put up. For now, just know that a task’s logs and effects are aggregated when getting run inside a job. Soon after a job is run, an archive is developed. An archive is a zipped folder made up of results information (XML/JSON formats), log information, and some added runtime data. These archives can be helpful for further results examination.

There are lots of extra components that make up pyATS, but these are some of the essential pieces. For additional information about the other factors, I hugely advocate examining out the pyATS documentation (link in the references).

USE Cases

To get your creative imagination flowing, let’s get a search at a couple of use situations that would be terrific matches for making use of pyATS.

1. Certifying a new network OS version
2. Validating operational condition of the network before/after a change
3. Managing intrusive exams to make sure network resilience

This record is not exhaustive and only utilised for demonstration. Let’s choose a swift look at just about every one particular.

CERTIFYING A Community OS Variation

A single of the worst items that can happen when you are upgrading products on your network to a new edition of computer software is managing into a bug. This bug might be clear or rooted deep in the OS and only induced when a precise function is configured. Regardless, administration and other stakeholders do not treatment that a bug was activated. They want to know why it wasn’t caught ahead of rolling out OS updates to production units. PyATS can supply a degree of certainty that a new OS edition works with the distinct hardware and program capabilities you have configured in your community. The pyATS screening framework can configure the functions you care about, validate every single feature’s features, and cleanup after tests has finished. It’s an automated approach that can immediately turn out to be a de facto course of action right before a new OS is rolled out to output.

VALIDATING OPERATIONAL Condition OF THE Network Ahead of/Soon after A Change

Validating variations on the network has been an difficulty as old as time. It’s a part of every engineer’s modify prepare but can at times be neglected when it will come to documentation. PyATS gives the framework to ensure the operational condition of your community and has developed-in reporting performance for you to rapidly figure out what validation checks have handed or failed. PyATS also provides extensive logging that captures unit logs, so you will be equipped to give all the correct documentation that reveals you confirmed the transform was prosperous.

Operating INTRUSIVE Checks TO Assure Community RESILIENCE

I would take into account this to be a more superior use situation, and one that should not be tried until eventually you have buy-in from better amounts of administration. Once you are comfy with managing examine-only tests versus your community, then you can start out introducing a minimal little bit of “chaos”. The appropriate title for this type of screening is “chaos engineering”. Netflix became well-known for using this practice as a result of a instrument they created called Chaos Monkey (https://netflix.github.io/chaosmonkey/). The plan is that random configuration is pushed to your creation setting to assure the infrastructure is resilient to failures. This random configuration may well incorporate shutting down BGP on a main router or rebooting a few products. Whatever the chaos may well be, the plan is to purposely cause failures in the infrastructure. You may be asking by yourself, “Why really should I carry out this kind of a cruel act against myself and my group?”. Well, the intention is for you to acquire publicity to the faults within just your community (and take care of them!) just before a real catastrophic failure happens.

Disclaimer: I’ve never experienced chaos engineering in a production environment. I incorporated this use situation to help showcase what’s doable the moment you’ve received self-confidence in your infrastructure, using automation.

SAMPLE CODE

In December 2022, I held an inside tech chat at NetCraftsmen about creating pyATS testscripts. In the demo, I crafted a pyATS testscript that incorporates a testcase for screening BGP. The BGP testcase contains the subsequent tests: look at for set up BGP neighbors, shut down BGP by shutting the WAN interface, test the routing desk for acquired BGP routes (really should be none), reactivate BGP, and last but not least, check out the routing table again for received BGP routes (ought to see BGP routes). The reason of this demo was to present how we can test BGP performance utilizing ‘show’ commands, even though transforming the test setting.

Here’s a url to the code repository: https://github.com/dannywade/20221215-pyATS-Testscripts

Sense absolutely free to open a Github issue to question questions or offer feedback.

WRAPPING UP

We went above a ton in this site post, and I certainly did not protect all the options of pyATS. The purpose of this submit was to contact on some of the higher-degree principles within the pyATS framework, and with any luck , get you wondering about how you can introduce automated community testing into your surroundings. If you are interested in automated community testing and not guaranteed exactly where to start out, be sure to experience free of charge to make contact with us and we can get you begun!

REFERENCES/Background Studying