Abstract

In modern software development, maintaining integration tests for web APIs often becomes a bottleneck. Traditional API test writing requires a large amount of boilerplate code: session setup, exception handling, data serialization, and so on. The “tests as data” approach offers a radically different philosophy, where a test is not a script but a description of what needs to be verified. Test suites written in imperative programming languages tend to become outdated faster than the codebase they test, because developers and testers lack the motivation to keep them up to date. Moreover, stakeholders without a programming background (e.g., analysts, manual QA specialists) find it difficult to understand such tests or contribute to them. This article explores whether a domain-specific language (DSL) can solve these problems.

Introduction

YATL is a DSL (domain-specific language) written in Python for API testing. It uses YAML as the test description language. But this is not just another framework for developers — it’s a tool that democratizes API testing, making it accessible to the entire team.

If you know HTTP and YAML — you know YATL.

Instead of writing imperative code, you declaratively describe tests in YAML files. Let’s take a look at what a simple GET request test looks like:

name: ping
base_url: google.com

steps:
- name: access_test
  request:
    method: GET
  expect:
    status: 200

No hidden “magic”: just a request (method, URL) and an expected response (status, partial body validation). This is much closer to a specification than to a script. Each .test.yaml file is a test scenario consisting of multiple steps.

Here’s what the same test would look like in Java:

import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.io.IOException;

public class HttpTest {

    public static void main(String[] args) {
        try {
            test();
        } catch (IOException | InterruptedException e) {
            e.printStackTrace();
        }
    }

    public static void test() throws IOException, InterruptedException {
        HttpClient client = HttpClient.newHttpClient();

        HttpRequest request = HttpRequest.newBuilder()
                .uri(URI.create("https://google.com"))
                .GET()
                .build();

        HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
        assert response.statusCode() == 200 : "Expected 200, got " + response.statusCode();

        System.out.println("All tests passed.");
    }
}

You’ll agree that the first option looks much friendlier and requires significantly less boilerplate.
Now let’s move on to the key features and highlights of the project:

Key Features

Support for All Data Formats

YATL understands most content types you’ll encounter in real-world scenarios right out of the box:

  • JSON — automatically sets Content-Type: application/json

  • XML — for SOAP and other XML-based APIs

  • Form data — application/x-www-form-urlencoded

  • Multipart files — file uploads

  • Plain text — text/plain

Example with JSON:

request:
  method: POST
  url: /users
  body:
    json:
      name: "Ivan Ivanov"
      email: "ivan@example.com"

Data Extraction and Templating

You can extract any fragments from responses and use them in subsequent requests via powerful Jinja2 templates. Dot notation is used to access nested JSON fields (e.g., user.info.name), keeping the syntax clean and concise. Consider this chain:

steps:
  - name: create_user
    request:
      method: POST
      url: /users
      body:
        json:
          name: "Alice"
    expect:
      status: 200
    extract:
      user_id: "response.id"  # Extract ID from response

  - name: get_user
    request:
      method: GET
      url: "/users/{{ user_id }}"   # Use the extracted ID     
      status: 200
      json:
        user.info.name: "Alice" # name validation via dot notation

Parallel Execution

By default, YATL runs tests in 10 threads (configurable via --workers). This dramatically speeds up running large test suites, such as regression tests.

Skipping Tests and Steps

In real-world development, you often need to temporarily disable a test without deleting it. YATL supports a skip mechanism:

name: Test in development
skip: true  # The entire test will be skipped

You can also skip an individual step:

steps:
  - name: active_step
    request: ...

  - name: skipped_step
    skip: true  # Only this step is skipped
    request: ...

CI/CD Integration

The true value of YATL is revealed when you integrate it into CI/CD.
This allows you to:

  • Catch bugs before they reach production — tests run on every PR
  • Never forget to run tests — the CI server handles it
name: API Tests
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-python@v4
        with:
          python-version: '3.14'
      - run: pip install yatl
      - run: yatl tests/ --workers 5

Important: YATL tests an already running API. It cannot start a server on its own. Therefore, in CI/CD, you must first start your API (with the latest code), and only then run yatl.

Test Parametrization

Sometimes you need to verify the same logic with different test data. For example, consider tests that check Google’s behavior:

name: ping
base_url: google.com

steps:
  - name: access_test
    request:
      method: GET
    expect:
      status: 200

  - name: failed_test
    request:
      method: GET
      url: /not_found
    expect:
      status: 404

You can notice code duplication — the tests differ only in the expected status and the URL. If we wanted to add new checks to the test (e.g., trigger a 400 error), we would have to add a new step. The same applies to deleting and editing test code. Many identical tests make test case maintenance harder. Let’s see how to avoid code duplication using parametrization.

Parametrization is a way to run the same test with different input data. In YATL, parametrization is implemented via the parametrize keyword. Here’s how to use it:

name: Params test
base_url: google.com

steps:
  - name: Ping

    parametrize:

      - path: /
        expected_status: 200

      - path: /not_found
        expected_status: 404

    request:
      method: GET
      url: "{{ path }}"
    expect:
      status: "{{ expected_status }}"

Why you should use parametrization:

  • Reduced duplication: test logic is defined once and executed with multiple parameters
  • Ease of maintenance: changes to test logic apply to all parameter combinations
  • Improved readability: clear separation between test structure and test data
  • Scalability: easy to add new test cases by adding parameter rows

Each test runs with its own parameters as a separate step:

Step 1, params: ('/', 200)
Step 2, params: ('/not_found', 404)

What’s Not Yet Available but Coming in the Release

Lifecycle hooks before / after (your suggestion!). This will allow you to automatically start and stop services directly from the test scenario. Here’s an example of how it will look:

name: My API tests
base_url: http://localhost:3000
before:
  command: "docker-compose up -d"
  wait_for: "http://localhost:3000/health"   # optional readiness check
after:
  command: "docker-compose down"
steps:
  - name: test endpoint
    request:
      method: GET
      url: /api

Data Validation

name: Extended validation example
steps:
  - name: get_user_data
    request:
      method: GET
      url: /api/users/123
    expect:
      status: 200 
      validate:
        - compare: { path: "user.age", gt: 18 }         
        - compare: { path: "user.name", min_length: 3 }     
        - compare: { path: "user.email", regex: ".+@.+\\..+" } 
        - compare: { path: "items", type: "array", not_empty: true }

Why This Isn’t Just Another Reinvention of the Wheel

Yes, there are Postman, Insomnia, Swagger UI, REST-assured, pytest + requests. But each of these tools has its own “Achilles’ heel”:

  • Postman — great for manual execution, but its collections are hard to version and difficult to integrate into CI (Newman is yet another layer).

  • pytest + requests — requires writing code, and as the number of tests grows, maintainability suffers.

  • Karate (Java DSL) — powerful, but still requires knowledge of the Java ecosystem.

YATL offers a universal test description language that is understandable to everyone on the team. An analyst can add a check for a new endpoint without bothering a developer. A QA engineer can reuse variables and schemas. And a developer can quickly debug a failing test just by looking at the YAML.

You can already install and try the library via the test package yatl-testing by running pip install yatl-testing.

By the way, starring the repo on GitHub is the best way to say “thank you”: repository link