Continuous Regression Testing for Engineering Teams

Touca provides feedback when you write code that could break your software. It remotely compares the behavior and performance of your software against a previous trusted version and visualizes differences in near real-time.

Start for free

Option 1: Self-host locally

You can self-host Touca by running the following command on a UNIX machine with at least 2GB of RAM, with Docker and Docker Compose installed.

/bin/bash -c "$(curl -fsSL https://touca.io/install.sh)"

Option 2: Use Touca Cloud

Or you can use https://app.touca.io that we manage and maintain. We have a free plan and offer additional features suitable for larger teams.

Sneak Peek

Touca has SDKs in Python, C++, Java, and JavaScript.

Let us imagine that we want to test a software workflow that takes the username of a student and provides basic information about them.

@dataclass
class Student:
    username: str
    fullname: str
    dob: datetime.date
    gpa: float

def find_student(username: str) -> Student:
    # ...

We can use unit testing in which we hard-code a set of usernames and list our expected return value for each input. In this example, the input and output of our code under test are username and Student. If we were testing a video compression algorithm, they may have been video files. In that case:

• Describing the expected output for a given video file would be difficult.
• When we make changes to our compression algorithm, accurately reflecting those changes in our expected values would be time-consuming.
• We would need a large number of input video files to gain confidence that our algorithm works correctly.

Touca makes it easier to continuously test workflows of any complexity and with any number of test cases.

import touca
from students import find_student

@touca.workflow
def students_test(username: str):
    student = find_student(username)
    touca.check("username", student.username)
    touca.check("fullname", student.fullname)
    touca.check("birth_date", student.dob)
    touca.check("gpa", student.gpa)

This is slightly different from a typical unit test:

• Touca tests do not use expected values.
• Touca tests do not hard-code input values.

With Touca, we describe how we run our code under test for any given test case. We can capture values of interesting variables and runtime of important functions to describe the behavior and performance of our workflow for that test case.

We can run Touca tests with any number of inputs from the command line:

touca config set api-key="<your_api_key>"
touca config set api-url="https://api.touca.io/@/tutorial"
touca test --revision=1.0 --testcase alice bob charlie

This command produces the following output:

Touca Test Framework

Suite: students_test/1.0

 1.  SENT   alice    (0 ms)
 2.  SENT   bob      (0 ms)
 3.  SENT   charlie  (0 ms)

Tests:      3 submitted, 3 total
Time:       0.39 s

✨   Ran all test suites.

Now if we make changes to our workflow under test, we can rerun this test and rely on Touca to check if our changes affect the behavior or performance of our software.

Unlike integration tests, we are not bound to the output of our workflow. We can capture any number of data points and from anywhere within our code. This is specially useful if our workflow has multiple stages. We can capture the output of each stage without publicly exposing its API. When any stage changes behavior in a future version of our software, our captured data points will help find the root cause more easily.

Value Proposition

Touca is very effective in addressing common problems in the following situations:

• When we need to test our workflow with a large number of inputs.
• When the output of our workflow is too complex, or too difficult to describe in our unit tests.
• When interesting information to check for regression is not exposed through the interface of our workflow.

The highlighted design features of Touca can help us test these workflows at any scale.

• Decoupling our test input from our test logic helps us manage our long list of inputs without modifying the test logic. Managing that list on a remote server accessible to all members of our team helps us add notes to each test case, explain why they are needed and track their stability and performance changes over time.
• Submitting our test results to a remote server, instead of storing them in files, helps us avoid the mundane tasks of managing and processing of test results. Touca server retains all test results and makes them accessible to all members of the team. It compares test results using their original data types and reports discovered differences in real-time to all interested members of our team. It helps us audit how our software evolves over time and provides high-level information about our tests.

Documentation

If you are new to Touca, the best place to start is our documentation website.

Community

We hang on Discord. Come say hi! We love making new friends. If you need help, have any questions, or like to contribute or provide feedback, that's the best place to be.

Contributors

Sponsors

License

This repository is released under the Apache-2.0 License. See LICENSE.