Open-source continuous regression testing that you can self-host.

Overview

Touca

Touca helps you see the side-effects of your changes, as you write code.

Touca Server

Touca SDKs let you describe the behavior and performance of your software workflows by capturing values of interesting variables and runtime of important functions. We remotely compare your description against a trusted version of your software, visualize all differences, and report them in near real-time.

C++ SDK Python SDK JavaScript SDK Java SDK

Links

Comments
  • JavaScript SDK should migrate away from Lerna

    JavaScript SDK should migrate away from Lerna

    ~We use Lerna to manage packages in our JavaScript SDK (see here). Lerna is now in a weird state (learn more here) and we like to remove it in favor of Nx or npm workspaces.~

    Requirements to be determined by @Rossh87 and @committomaster.

    ✨ Good First Issue 💪🏼 Improvement Build System JavaScript SDK 
    opened by ghorbanzade 20
  • build: reduce docker image size (#311)

    build: reduce docker image size (#311)

    Final image size: 450MB -> 168MB Build time (without cache) on my machine: basically equal

    Overview: In builder_api stage, install all dependencies for the 4 needed modules, build them, recursively delete all node_modules, and re-install with --prod. Setting --ignore-scripts on install disables the postinstall scripts on packages in <repository>/packages/*, since their build step is now handled by pnpm. Without this, the image will fail to build, since packages/* will try to rebuild themselves with devDependencies that are no longer installed.

    QA: I was able to build the image locally and run it against local Mongo/Redis/MinIO instances without any issue. I ran through the onboarding flow with the web app running against this container, and loaded sample data, also without issue. I was not able to get the server to load .env.production with docker run --env environment=production, and so I wasn't able to test that the app bundle was being served correctly, which I assume is important for this container. There's probably something obvious I'm missing. Let me know if there's a good way for me to check this, or if there's anything else I can change/provide!

    opened by Rossh87 11
  • Move client to an event-based update model

    Move client to an event-based update model

    (moving to Github from this discussion)

    Description

    Angular app client currently polls API server for data updates. Pages re-request all their data every 10 seconds. This change would move the client to a server event based model, minimizing redundant network traffic and improving the client's response time to server-side changes.

    Suggested approach

    Send events to authenticated clients whenever Redis cache elements are invalidated/removed, and let the client decide whether or not to refetch the impacted data. I favor this approach for two reasons:

    1. It's simpler to implement/extend than trying to proactively send updated data in the event itself. It's easier to reason about potential race conditions, and doesn't require any fussy data merging on the client side.
    2. It avoids broadcasting useless data to clients that don't need it.
    3. Would play nicely with client-side caching if you ever wanted to add it.

    Drawbacks

    1. Will cause a spike of HTTP requests whenever a cache invalidation occurs. I think this can be mitigated by adding a bit of delay before the server event is sent to too many clients; that should ensure the cache is repopulated before the bulk of the requests come through.
    2. In some cases this approach will cause more network and database usage than including data in the server event would. For example, if two clients are viewing the batch list page, and one of them submits a new batch, it is more expensive to refetch all the relevant batches again than it would be to append the new batch to the list that the clients already have in memory. I think points 1 and 2 above outweigh this concern, but others might reasonably disagree. At worst, taking the cache invalidation approach outlined above doesn't preclude using a data broadcasting strategy for cases where a complete refetch would be too expensive.

    Questions

    1. Do you think there will ever be value in having the two-way communication that websockets offer? If not, I would favor using server events, since they're simple enough to implement without adding any third-party dependencies.
    2. Do you/would you like to support scenarios where multiple instances of the Touca API server run as part of the same service, e.g. behind a load balancer? If so, push events will need to go through the async queue.

    Sorry for writing a novel. I'm always happy to have my mind changed, and I really welcome any suggestions or criticisms before getting started!

    opened by Rossh87 6
  • Install script creates some local files as root

    Install script creates some local files as root

    Description

    When running install.sh, Ehsan Ahangari found that files in directories like /.touca/server/data/mongo and /.touca/server/data/minio are installed as owned by root instead of the current user.

    Environment

    Checked on Rocky Linux and Ubuntu

    🐛 Bug Build System 
    opened by ghorbanzade 5
  • Share types between Touca Server components

    Share types between Touca Server components

    Currently we have two separate files api/src/types/commontypes.ts and app/src/app/core/models/commontypes.ts with duplicate content that allow Touca frontend understand API response from the backend.

    We'd like to create a private npm package @touca/server-types so that we can share these TypeScript types without duplicating them.

    ✨ Good First Issue 💪🏼 Improvement Web App Server API 
    opened by ghorbanzade 2
  • Touca Cloud Web App shows failure when creating suite

    Touca Cloud Web App shows failure when creating suite

    Description

    Creating a new suite results in the error: 'Something went wrong. Please try this operation again at a later time' despite the suite being created.

    Environment

    I'm using ubuntu 18.04.05

    Steps To Reproduce

    Create a new suite

    🐛 Bug Web App Server API 
    opened by rmarrcode 2
  • sdk-js: sub npm workspaces for lerna (#193)

    sdk-js: sub npm workspaces for lerna (#193)

    QA:

    • npm run <cmd> work as expected for all scripts in /sdk/js/package.json
    • all tests in /sdk/js/packages/node/test passing
    • build takes a bit longer since builds no longer run in parallel. No effect on build:prod since examples don't build.
    opened by Rossh87 1
  • Web app has no way to switch teams on small screens

    Web app has no way to switch teams on small screens

    Description

    There is no way for a user to switch teams when using the web app on small screens.

    Expected Behavior

    Web App's teams page should include a drop-down button (much like the Server Settings page) that allows users to switch between their teams.

    🐛 Bug Web App 
    opened by ghorbanzade 1
  • Add micro benchmarks for comparison functions in C++ SDK

    Add micro benchmarks for comparison functions in C++ SDK

    Touca's cpp-sdk is subject to performance and layout improvements (for example as #107 suggests) and features like supporting blobs as data_point are in discussion.

    As performance of changes aren't measured yet, these changes and their impact aren't quite comparable, thus there is a need of adding benchmarking.

    💪🏼 Improvement C++ SDK 💡 Community Inspired 
    opened by cthulhu-irl 1
  • Server-sent events may not work correctly if multiple server instances are running

    Server-sent events may not work correctly if multiple server instances are running

    Description

    Delivery of server-sent events to clients will become unreliable if more than 1 instance of touca-app is running as a cluster/single service. This may not be an important configuration to support, but I thought I'd flag it. Feel free to close if not relevant.

    Environment

    Any configuration that can deploy multiple instances of the server container, e.g. K8s, Docker Swarm.

    Steps To Reproduce

    Deploy as per Environment above, with at least 2 instances of touca-app running. Connect multiple clients to touca-app. Complete an action from one of the clients that should trigger the delivery of an event to all connected clients. The event may or may not be broadcast to all appropriate clients.

    Expected Behavior

    All clients should receive all relevant events, regardless of which instance of touca-app they are connected to.

    Additional Context

    Not a high-priority fix, but it will definitely come up if you ever need to horizontally scale Touca.

    It happens because BullMQ does not support a 'fanout' job distribution pattern, and it doesn't seem like they will any time soon. Since each instance of touca-app keeps track of server-event subscriptions in its own process, and each enqueued job is consumed by exactly one worker and then discarded, only the clients who happen to be connected to the same process as the worker that consumes the job can receive the event.

    Low Priority 
    opened by Rossh87 2
  • Remove instances of `as unknown as` from server code

    Remove instances of `as unknown as` from server code

    As part of #204, we introduced a few instance of as unknown as to the code to convert from the generated types of @touca/api-schema to other types. A majority of these instances are cases when the schema.yaml file ignores format: date-time and creates strings for Date types. We like to find a way to remove or drastically reduce these instances.

    💪🏼 Improvement Server API 
    opened by ghorbanzade 0
  • Split JavaScript SDK into separate packages

    Split JavaScript SDK into separate packages

    Currently, our JavaScript SDK comes in one single package @touca/node with dependencies on fs, path, http, https, and url node modules. This prohibits using the SDK in environments where node is not available. In addition, the SDK comes includes a test framework that is responsible for taking command line arguments and reporting the outcome of the test through the standard output. We rely on dependencies such as yargs and chalk to perform these operations.

    We'd like to split @touca/node into packages @touca/core and @touca/cli and move the test framework and other extra SDK functionalities into the @touca/cli package, making @touca/core suitable for a variety of environments.

    ✨ Good First Issue ✨ New Feature Build System JavaScript SDK 
    opened by ghorbanzade 4
  • Find a better way to run Java example

    Find a better way to run Java example

    Our Java Examples in examples/java and sdk/java directories are defined as standalone applications. Unlike JUnit tests that are in often placed in the src/testdirectory of a given library/package, Touca tests live outside the library in a separate examples directory. This is unintuitive. But we are no Java or Gradle experts and simply didn't know any better. If you have ideas on how to fix this, please have at it.

    Because of this directory layout, we reference these Java examples in settings.gradle.kts and have separate build.gradle.kts files for each example. Each build file has a JavaExec task like below:

    task<JavaExec>("runExampleMinimal") {
        main = "io.touca.examples.minimal.PrimeTest"
        classpath = sourceSets["main"].runtimeClasspath
    }
    

    This way, we can run the example application using the following command:

    gradle runExampleMinimal --args='--api-key <TOUCA_API_KEY> --api-url <TOUCA_API_URL> --revision 1.0 --testcase 13'
    

    Note that we are using gradle here instead of gradlew so we can run the task and pass the appropriate command line arguments. Very ugly. Yuck! There must be a better way. We don't know it yet, but maybe you do. The first task for this issue is to figure out how to improve this setup. The second task is to actually implement it and convert the existing examples to the new pattern accordingly. We should also update all getting started documents and markdown files in examples directory as part of this work.

    ✨ Good First Issue 💪🏼 Improvement Java SDK 
    opened by ghorbanzade 0
Releases(v1.7.0)
  • v1.7.0(Sep 21, 2022)

    What's Changed

    Server

    • use structured logs in cloud mode (#343)
    • fix e2e setup code for clearing buckets (#342)
    • fix race condition in message ingestion logic (#341)
    • sort rows in account list (#325)
    • continue applying mail server env vars (#322)
    • remove deprecated fields from database (#321)
    • use uuid from meta when reporting self-hosted installs (#320)
    • add separate route to relay self-hoted installs (#319)
    • change feedback processing route (#318)
    • add route to relay telemetry (#316)
    • update root url in cloud hosted mode (#313)
    • skip serving static files in cloud-hosted mode (#312)
    • do not create log file by default (#309)
    • report perfect score when comparing empty messages (#307)
    • fix status check logic in install script (#302)
    • convert server to single docker image (#289)
    • remove comparator component (#267)

    Documentation Website

    • consolidate quickstart files (#328)
    • add instructions for new CLI subcommand results (#326)

    Marketing Website

    • publish new changelog (#327)
    • publish new changelog (#315)

    Python SDK

    • add new CLI sub-command result (#324)

    JavaScript SDK

    • bump development version to v1.5.7 (#339)
    • support configuration profiles (#338)
    • bump development version to v1.5.6 (#337)
    • add support for automatic version increments (#336)
    • change default output directory (#333)
    • enable running multiple workflows (#332)
    • update dependencies (#329)

    Build System

    • change public ecr repo for docker image (#310)
    • update manifest and helm chart to use v1.7 (#304)
    • bump server to v1.7.0 (#299)

    Full Changelog: https://github.com/trytouca/trytouca/compare/v1.6.0...v1.7.0

    Source code(tar.gz)
    Source code(zip)
  • v1.6.0(Sep 7, 2022)

    What's Changed

    Server

    • fix comparison of empty objects (#296)
    • add new comparison logic (#294)
    • add analytics for team member management (#290)
    • reformat index html files (#284)
    • create account during install wizard (#283)
    • refactor analytics collection (#282)
    • simplify analytics collection (#281)
    • integrate customer.io (#276)
    • add experimental comparison service (#266)

    Marketing Website

    • publish new changelog (#292)
    • remove intercom (#280)
    • publish new changelog (#275)
    • add intercom (#269)

    Python SDK

    • improved progress update logic in cli subcommand zip (#291)
    • refactor cli subcommand zip (#288)
    • add default src directory for cli subcommand post (#287)
    • refactor cli subcommand post (#286)
    • add help command (#279)
    • remove init from api reference docs (#278)
    • use rich text for config, profile, and plugin CLI commands (#277)
    • use next batch if not specified (#274)
    • bump development version to v1.5.8 (#268)
    • add release notes for v1.5.7 (#265)

    C++ SDK

    • enable deserializing results with no team slug (#285)

    Build System

    • remove unnecessary git fetch tags (#298)
    • ci should build api when packages are changed (#297)
    • ci should build comparator package (#295)
    • change path to external docs (#273)
    • remove ci step to deploy ops directory (#272)
    • add build-ops job to ci (#271)
    • add helm chart and manifest files for kubernetes deployment (#270)
    • bump development version to v1.6.0 (#264)
    • bump server to v1.5.0 (#263)

    New Contributors

    • @ehsan-touca made their first contribution (#270)

    Full Changelog: https://github.com/trytouca/trytouca/compare/v1.5.0...v1.6.0

    Source code(tar.gz)
    Source code(zip)
  • v1.5.0(Aug 20, 2022)

    What's Changed

    Server

    • update dependencies (https://github.com/trytouca/trytouca/pull/165)
    • remove aws creds for pdf generation (https://github.com/trytouca/trytouca/pull/169)
    • simplify api env files (https://github.com/trytouca/trytouca/pull/170)
    • use aws-sdk in backend to connect to object storage (https://github.com/trytouca/trytouca/pull/171)
    • comparator should bypass proxy_host when connecting to s3 (https://github.com/trytouca/trytouca/pull/174)
    • avoid using list buckets in comparator (https://github.com/trytouca/trytouca/pull/175)
    • determine object_store connection based on host (https://github.com/trytouca/trytouca/pull/176)
    • use credentials for s3 connection (https://github.com/trytouca/trytouca/pull/177)
    • remove use of let in aggregate pipeline (https://github.com/trytouca/trytouca/pull/178)
    • switch DB query for fetching comparison jobs (https://github.com/trytouca/trytouca/pull/180)
    • install apt-transport-https in cmp docker (https://github.com/trytouca/trytouca/pull/181)
    • set default values for some env variables (https://github.com/trytouca/trytouca/pull/184)
    • switch to using pnpm (https://github.com/trytouca/trytouca/pull/187)
    • share types between server components by @committomaster (https://github.com/trytouca/trytouca/pull/198)
    • make npm scripts cross-platform by @committomaster (https://github.com/trytouca/trytouca/pull/200)
    • add build script to api-schema package (https://github.com/trytouca/trytouca/pull/202)
    • autogenerate types using OpenAPI schema by @committomaster (https://github.com/trytouca/trytouca/pull/204)
    • update dockerfiles and related scripts (https://github.com/trytouca/trytouca/pull/207)
    • e2e test should retry server connection (https://github.com/trytouca/trytouca/pull/214)
    • split install page to smaller components (https://github.com/trytouca/trytouca/pull/219)
    • jwt tokens should always a buffer until expiration time (https://github.com/trytouca/trytouca/pull/225)
    • move flatbuffers schema definition to separate package (https://github.com/trytouca/trytouca/pull/226)
    • rewrite telemetry tab of install workflow (https://github.com/trytouca/trytouca/pull/236)
    • add telemetry service (https://github.com/trytouca/trytouca/pull/237)
    • make teams page width consistent with other pages (https://github.com/trytouca/trytouca/pull/238)
    • log mongo_uri during startup (https://github.com/trytouca/trytouca/pull/244)
    • add settings tab for configuring mail transport (https://github.com/trytouca/trytouca/pull/247)
    • fix broken pdf export feature (https://github.com/trytouca/trytouca/pull/249)
    • disable mail server when running e2e test (https://github.com/trytouca/trytouca/pull/251)
    • do not show scrollbars for pills (https://github.com/trytouca/trytouca/pull/253)
    • show version in help component (https://github.com/trytouca/trytouca/pull/254)

    Documentation

    • add openbase badge to readme (https://github.com/trytouca/trytouca/pull/164)
    • fix broken links in clone commands (https://github.com/trytouca/trytouca/pull/183)
    • add new categories to sideline menu (https://github.com/trytouca/trytouca/pull/188)
    • add contribution guidelines (https://github.com/trytouca/trytouca/pull/197)
    • update list of contributors (https://github.com/trytouca/trytouca/pull/201)
    • add ideal image plugin (https://github.com/trytouca/trytouca/pull/206)
    • remove outdated pricing page (https://github.com/trytouca/trytouca/pull/210)
    • update dependencies (https://github.com/trytouca/trytouca/pull/227)
    • fixed typo in getting started page by @DepthDeluxe (https://github.com/trytouca/trytouca/pull/242)
    • enable search using algolia (https://github.com/trytouca/trytouca/pull/246)
    • use ideal image plugin in mdx files (https://github.com/trytouca/trytouca/pull/255)
    • add page env variables supported by the server (https://github.com/trytouca/trytouca/pull/256)
    • add helper react component ToucaImage (https://github.com/trytouca/trytouca/pull/257)
    • fix analytics script (https://github.com/trytouca/trytouca/pull/258)
    • add separate page for cpp cli (https://github.com/trytouca/trytouca/pull/259)
    • update cli commands (https://github.com/trytouca/trytouca/pull/260)
    • use custom image component in all pages (https://github.com/trytouca/trytouca/pull/261)

    Marketing Website

    • update landing page (https://github.com/trytouca/trytouca/pull/209)
    • remove announcement from header (https://github.com/trytouca/trytouca/pull/213)
    • change above-the-fold section (https://github.com/trytouca/trytouca/pull/215)
    • update above-the-fold copy (https://github.com/trytouca/trytouca/pull/182)
    • update blog content logic (https://github.com/trytouca/trytouca/pull/232)
    • add separate changelog page (https://github.com/trytouca/trytouca/pull/233)
    • add risk mitigations to above-the-fold section (https://github.com/trytouca/trytouca/pull/241)
    • publish new changelog (https://github.com/trytouca/trytouca/pull/243)
    • fix atf section on small screens (https://github.com/trytouca/trytouca/pull/248)
    • publish new changelog (https://github.com/trytouca/trytouca/pull/262)

    Python SDK

    • update readme file (https://github.com/trytouca/trytouca/pull/173)
    • CLI post sub-command should not expect merged files by @mmdbalkhi (https://github.com/trytouca/trytouca/pull/217)
    • CLI post sub-command should respect config profile by @mmdbalkhi (https://github.com/trytouca/trytouca/pull/222)
    • remove default API URL (https://github.com/trytouca/trytouca/pull/223)
    • refactor dry-run in post sub-command (https://github.com/trytouca/trytouca/pull/224)
    • remove loguru as dependency (https://github.com/trytouca/trytouca/pull/229)
    • update supported options for CLI sub-command profile (https://github.com/trytouca/trytouca/pull/239)
    • replace CLI sub-command solve with server (https://github.com/trytouca/trytouca/pull/231)

    JavaScript SDK

    • switch to using npm (https://github.com/trytouca/trytouca/pull/185)
    • upgrade development version to v1.5.4 (https://github.com/trytouca/trytouca/pull/199)

    Build System

    • use docker/metadata action in ci (https://github.com/trytouca/trytouca/pull/163)
    • connect to AWS using OpenID Connect (https://github.com/trytouca/trytouca/pull/166)
    • simplify aws secrets (https://github.com/trytouca/trytouca/pull/167)
    • update dependencies for web and docs (https://github.com/trytouca/trytouca/pull/168)
    • disable building cmp docker for arm64 (https://github.com/trytouca/trytouca/pull/172)
    • unpin gcovr package to use latest version (https://github.com/trytouca/trytouca/pull/228)
    • fix broken self-hosted install (https://github.com/trytouca/trytouca/pull/230)
    • update dev image for api (https://github.com/trytouca/trytouca/pull/245)
    • find version from package.json (https://github.com/trytouca/trytouca/pull/252)

    New Contributors

    • @committomaster made their first contribution (https://github.com/trytouca/trytouca/pull/198)
    • @mmdbalkhi made their first contribution (https://github.com/trytouca/trytouca/pull/217)
    • @DepthDeluxe made their first contribution (https://github.com/trytouca/trytouca/pull/242)

    Full Changelog: https://github.com/trytouca/trytouca/compare/v1.4.0...v1.5.0

    Source code(tar.gz)
    Source code(zip)
Owner
Touca
See the side effects of your changes, as you write code.
Touca
DropSpace is an easy way to self-host a file drop.

DropSpace Simple file sharing made in Laravel About DropSpace DropSpace is an easy way to self-host a file drop. Simply upload your file, and share th

Kazó Levente 25 Dec 9, 2022
This is a library to alternate and self-host the Prisma Data Proxy (cloud.prisma.io)

Alternative Prisma Data Proxy This is a library to alternate and self-host the Prisma Data Proxy (cloud.prisma.io). In order to deploy your project to

AijiUejima 60 Dec 28, 2022
Esther Udondian 6 Aug 1, 2022
A testing focused Remix Stack, that integrates E2E & Unit testing with Playwright, Vitest, MSW and Testing Library. Driven by Prisma ORM. Deploys to Fly.io

Live Demo · Twitter A testing focused Remix Stack, that integrates E2E & Unit testing with Playwright, Vitest, MSW and Testing Library. Driven by Pris

Remix Stacks 18 Oct 31, 2022
Continuous Deployment Pipelines (CI/CD) with Dozer

Continuous deployment pipelines are sets of scripts and tools helping you get your project from your workstation, or version control system, to somewhere your target audience can reach it.

Mirek Kaspar 13 Jan 2, 2023
A tutorial for people that want to learn continuous delivery concepts by practicing.

Continuous delivery playground This tutorial walks you through setting up a continuous delivery pipeline from scratch. This guide is for people that w

Pablo Porto 7 Nov 24, 2022
A timed continuous carousel that uses vanilla JavaScript & CSS animations.

Continuous Carousel ∞ Timed continuous carousel that uses vanilla JavaScript & CSS animations. It supports both horizontal and vertical scrolling. Mar

Jon C. 16 Jan 3, 2023
Application for self-testing before exams covering some of the subject taught at the Prague University of Economics and Business

Getting Started with Create React App This project was bootstrapped with Create React App. Available Scripts In the project directory, you can run: ya

Jiří Vrba 2 Jun 13, 2022
An open source, self-hosted, and entirely free solution to social media management.

An open source, self-hosted, and entirely free solution to social media management. Status ?? In Development ?? Shoutify is currently early in the dev

TechSquidTV 202 Dec 22, 2022
Self-hosted open source social media marketing

OpenSMM Description OpenSMM(Open Social Media Marketing) is a self-hosted social media marketing platform built to assist small businesses manage thei

Vaughn 22 Dec 23, 2022
A free, open-source, web-based self-service BI tailor-made for analytical databases.

Rocket BI RocketBI is a self-service, web-based business intelligent product tailor-made for analytical databases. RocketBI is the core product of Dat

Data Insider 19 Jan 3, 2023
An open-source and self-hosted assessment platform for recruitment

This is a Next.js project bootstrapped with create-next-app. Getting Started First, run the development server: npm run dev # or yarn dev Open http://

Kaen 2 Oct 4, 2022
Javascript-testing-practical-approach-2021-course-v3 - Javascript Testing, a Practical Approach (v3)

Javascript Testing, a Practical Approach Description This is the reference repository with all the contents and the examples of the "Javascript Testin

Stefano Magni 2 Nov 14, 2022
AREX: It is a “Differential Testing” and “Record and Replay Testing” Tool.

AREX: It is a “Differential Testing” and “Record and Replay Testing” Tool. Test restful API by record, replay and stub request/response. Differential

ArexTest 15 Nov 1, 2022
An Open Source Remix template that integrates Stripe Subscriptions, Social Authentication, Testing and a few more features. PostgreSQL version. Deploys to Fly.io

Live Demo · Twitter An open source Remix Stack that integrates Stripe Subscriptions, Social Authentication, Testing and a few more features. PostgreSQ

xo 25 Dec 7, 2022
An Open Source Remix template that integrates Stripe Subscriptions, Social Authentication, Testing and a few more features. SQLite version. Deploys to Fly.io

Live Demo · Twitter An Open Source Remix template that integrates Stripe Subscriptions, Social Authentication, Testing and a few more features. SQLite

xo 135 Dec 31, 2022
Reference for How to Write an Open Source JavaScript Library - https://egghead.io/series/how-to-write-an-open-source-javascript-library

Reference for How to Write an Open Source JavaScript Library The purpose of this document is to serve as a reference for: How to Write an Open Source

Sarbbottam Bandyopadhyay 175 Dec 24, 2022