# Testing

> How to run unit and blackbox tests in the Directus codebase.

The current test strategy for Directus consists of blackbox tests, which test the overall functionality of the platform, as well as unit tests, which test individual parts of the codebase.

## Running Unit Tests

Use the following command to perform unit tests in all packages:

```bash
pnpm --workspace-root test
```

Use one of the following commands to perform more specific actions with unit tests (mix and match as desired):

```bash
# Run tests for a specific package (for example only in the api or app package)
pnpm --filter api test
pnpm --filter app test

# Start tests in watch mode
pnpm --filter api test -- --watch

# Enable coverage report
pnpm --filter api test -- --coverage

# Run specific test files using a filter pattern
pnpm --filter api test -- app.test.ts
pnpm --filter api test -- utils
```

<callout icon="i-lucide-info">

**Relative Commands**

If you are already in a directory of a specific package, you may omit the `--filter` flag in `pnpm` commands since the commands will be executed relative to the current directory.

```bash
# Run API tests, from within the "/api" directory
pnpm test
```

</callout>

## Running Blackbox Tests

Install [Docker](https://docs.docker.com/get-docker/) and ensure that the service is up and running. Run the following commands to start the blackbox tests:

```bash
# Ensure that you are testing against the latest state of the codebase
pnpm --workspace-root build

# Clean up in case you ran the tests before
pnpm --filter tests-blackbox exec docker compose down --volumes
# Start the containers required for the tests
pnpm --filter tests-blackbox exec docker compose up --detach --wait

# Deploy Directus and run the tests
## Run common tests unrelated to database
pnpm --workspace-root test:blackbox -- --project common
## Run database specific tests
pnpm --workspace-root test:blackbox -- --project db
```

Subsequent test runs can be issued with the following command, if only modifications to the blackbox tests themselves have been made:

```bash
## Run common tests unrelated to database
pnpm --filter tests-blackbox test --project common
## Run database specific tests
pnpm --filter tests-blackbox test --project db
```

### Testing Specific Database Vendors

Provide a comma-separated list of database vendors via the `TEST_DB` environment variable to target only a specific subset:

```bash
# Example targeting multiple vendors
TEST_DB=cockroachdb,postgres pnpm --workspace-root test:blackbox -- --project db

# Example targeting a single vendor
TEST_DB=sqlite3 pnpm --workspace-root test:blackbox -- --project db
```

If tests are only run against a subset of databases, it also makes sense to only start the corresponding containers:

```bash
# Start the containers that are always required
pnpm --filter tests-blackbox exec docker compose up auth-saml redis minio minio-mc --detach --wait

# Start the specific database container (for example 'postgres')
pnpm --filter tests-blackbox exec docker compose up postgres --detach --wait
```

### Using an Existing Directus Project

Usually, the test suite will spin up a fresh copy of the Directus API built from the current state of the codebase. To use an already running instance of Directus instead, enable the `TEST_LOCAL` flag:

```bash
TEST_DB=cockroachdb TEST_LOCAL=true pnpm --workspace-root test:blackbox -- --project db
```

Note: The tests expect the instance running at `localhost:8055`. Make sure to connect the instance to the test database container found in the `tests/blackbox/docker-compose.yml` file.

### Server Logs

For debugging purposes, server logs can be enabled by specifying a log level using the `TEST_SAVE_LOGS` flag, for example:

```bash
TEST_SAVE_LOGS=info pnpm --workspace-root test:blackbox -- --project db
```

The log files will be available under `tests/blackbox/server-logs-*`.

## Writing Unit Tests

Unit Tests are written throughout the codebase in a vite native unit test framework called [Vitest](https://vitest.dev).

### Example

```ts [/directus/api/src/utils/get-date-formatted.test.ts]
import { afterEach, beforeEach, expect, test, vi } from 'vitest';

import { getDateFormatted } from './get-date-formatted.js';

beforeEach(() => {
    vi.useFakeTimers();
});

afterEach(() => {
    vi.useRealTimers();
});

function getUtcDateForString(date: string) {
    const now = new Date(date);

    // account for timezone difference depending on the machine where this test is ran
    const timezoneOffsetInMinutes = now.getTimezoneOffset();
    const timezoneOffsetInMilliseconds = timezoneOffsetInMinutes * 60 * 1000;
    const nowUTC = new Date(now.valueOf() + timezoneOffsetInMilliseconds);

    return nowUTC;
}

test.each([
    { utc: '2023-01-01T01:23:45.678Z', expected: '20230101-12345' },
    { utc: '2023-01-11T01:23:45.678Z', expected: '20230111-12345' },
    { utc: '2023-11-01T01:23:45.678Z', expected: '20231101-12345' },
    { utc: '2023-11-11T12:34:56.789Z', expected: '20231111-123456' },
    { utc: '2023-06-01T01:23:45.678Z', expected: '20230601-12345' },
    { utc: '2023-06-11T12:34:56.789Z', expected: '20230611-123456' },
])('should format $utc into "$expected"', ({ utc, expected }) => {
    const nowUTC = getUtcDateForString(utc);

    vi.setSystemTime(nowUTC);

    expect(getDateFormatted()).toBe(expected);
});
```

## Writing Blackbox Tests

### Example

```ts [/directus/tests/blackbox/routes/server/ping.test.ts]
import { getUrl } from '@common/config';
import request from 'supertest';
import vendors from '@common/get-dbs-to-test';
import { requestGraphQL } from '@common/transport';

describe('/server', () => {
    describe('GET /ping', () => {
        it.each(vendors)('%s', async (vendor) => {
            // Action
            const response = await request(getUrl(vendor))
                .get('/server/ping')
                .expect('Content-Type', /text\/html/)
                .expect(200);

            const gqlResponse = await requestGraphQL(getUrl(vendor), true, null, {
                query: {
                    server_ping: true,
                },
            });

            // Assert
            expect(response.text).toBe('pong');
            expect(gqlResponse.body.data.server_ping).toBe('pong');
        });
    });
});
```