Skip to content

use set up and tear down test db during backend tests#3038

Open
danoswaltCL wants to merge 1 commit intodevfrom
fix/integration-test-config-cleanup
Open

use set up and tear down test db during backend tests#3038
danoswaltCL wants to merge 1 commit intodevfrom
fix/integration-test-config-cleanup

Conversation

@danoswaltCL
Copy link
Collaborator

this will create a separate test db when running backend tests, as it currently uses the same as local dev and clobbers that data without cleanup, we don't have to live this way

also cleans out some unneeded harcoded users and vivek email address

hopefully this works on jenkins, I guess we'll see. if not, will see where to conditionally make sure cicd still uses postgres for the tests.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds integration-test DB lifecycle management so backend integration tests can run against an isolated Postgres database (instead of clobbering local dev data), and removes hardcoded personal emails/users from test fixtures/config.

Changes:

  • Add Jest global setup/teardown scripts to create/drop the integration test database.
  • Introduce a dedicated Jest integration config and update integration test scripts to use it.
  • Remove hardcoded user/email references in integration test data and delete src/config.json.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
packages/backend/test/integration/setup/globalSetup.js Creates the test database before integration suites run.
packages/backend/test/integration/setup/globalTeardown.js Terminates connections and drops the test database after integration suites run.
packages/backend/jest.integration.config.js New integration-only Jest config (currently only narrows test matching).
packages/backend/jest.config.js Enables Jest global setup/teardown hooks (currently applied globally).
packages/backend/package-scripts.js Updates integration test commands to use jest.integration.config.js.
packages/backend/test/integration/mockData/user/index.ts Replaces hardcoded personal email/name with mock values.
packages/backend/test/integration/Experiment/analytics/MonitoredPointForExport.ts Uses systemUser.email instead of a hardcoded email string.
packages/backend/src/config.json Removes a backend config file containing hardcoded user list/secrets.
packages/backend/.gitignore Un-ignores integration setup JS files so they can be committed.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

const path = require('path');

module.exports = async function globalSetup() {
dotenv.config({ path: path.join(process.cwd(), '.env.test') });
Copy link

Copilot AI Mar 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using dotenv.config({ path: path.join(process.cwd(), '.env.test') }) makes setup dependent on the process working directory (e.g., running Jest from the repo root with --config packages/backend/jest.integration.config.js won’t find the env file). Prefer resolving relative to this file (via __dirname) or Jest’s <rootDir> so the env file path is stable across local/CI execution contexts.

Suggested change
dotenv.config({ path: path.join(process.cwd(), '.env.test') });
dotenv.config({ path: path.join(__dirname, '../../../.env.test') });

Copilot uses AI. Check for mistakes.
module.exports = async function globalSetup() {
dotenv.config({ path: path.join(process.cwd(), '.env.test') });

const testDb = process.env.TYPEORM_DATABASE;
Copy link

Copilot AI Mar 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TYPEORM_DATABASE is used to determine which database to create, but it isn’t validated. Add guards to ensure it’s defined, not postgres, and matches a safe identifier pattern (e.g. /^[A-Za-z0-9_]+$/) before it’s later interpolated into CREATE DATABASE "${testDb}", to avoid targeting/dropping the wrong DB and to prevent identifier-injection via env vars.

Suggested change
const testDb = process.env.TYPEORM_DATABASE;
const testDb = process.env.TYPEORM_DATABASE;
const dbNamePattern = /^[A-Za-z0-9_]+$/;
if (!testDb) {
throw new Error('TYPEORM_DATABASE environment variable must be defined for test setup');
}
if (testDb.toLowerCase() === 'postgres') {
throw new Error('TYPEORM_DATABASE must not be set to "postgres" for test setup');
}
if (!dbNamePattern.test(testDb)) {
throw new Error('TYPEORM_DATABASE contains invalid characters; only A-Za-z0-9_ are allowed');
}

Copilot uses AI. Check for mistakes.
Comment on lines +18 to +22
const result = await client.query('SELECT 1 FROM pg_database WHERE datname = $1', [testDb]);
if (result.rowCount === 0) {
await client.query(`CREATE DATABASE "${testDb}"`);
console.log(`Created test database: ${testDb}`);
}
Copy link

Copilot AI Mar 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a previous test run exits before globalTeardown (e.g., SIGKILL/CI timeout), the test DB will remain and globalSetup will currently reuse it without resetting state (it only creates if missing). To guarantee a clean run, consider dropping/recreating the DB in setup (after terminating connections) or otherwise clearing schema/data deterministically.

Suggested change
const result = await client.query('SELECT 1 FROM pg_database WHERE datname = $1', [testDb]);
if (result.rowCount === 0) {
await client.query(`CREATE DATABASE "${testDb}"`);
console.log(`Created test database: ${testDb}`);
}
// Ensure no active connections to the test database from previous runs
await client.query(
'SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE datname = $1 AND pid <> pg_backend_pid()',
[testDb],
);
// Drop and recreate the test database to guarantee a clean state
await client.query(`DROP DATABASE IF EXISTS "${testDb}"`);
await client.query(`CREATE DATABASE "${testDb}"`);
console.log(`Recreated test database: ${testDb}`);

Copilot uses AI. Check for mistakes.
Comment on lines +4 to +6

module.exports = async function globalTeardown() {
dotenv.config({ path: path.join(process.cwd(), '.env.test') });
Copy link

Copilot AI Mar 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like setup, teardown relies on process.cwd() to locate .env.test, which can break when Jest is invoked from a different working directory. Resolve the env file path relative to the repo/workspace instead of the current process CWD to make teardown reliable in CI and when running from the monorepo root.

Suggested change
module.exports = async function globalTeardown() {
dotenv.config({ path: path.join(process.cwd(), '.env.test') });
const fs = require('fs');
function findEnvTest(startDir) {
let dir = startDir;
while (true) {
const candidate = path.join(dir, '.env.test');
if (fs.existsSync(candidate)) {
return candidate;
}
const parent = path.dirname(dir);
if (parent === dir) {
break;
}
dir = parent;
}
return null;
}
module.exports = async function globalTeardown() {
const envPath = findEnvTest(__dirname);
if (envPath) {
dotenv.config({ path: envPath });
} else {
dotenv.config();
}

Copilot uses AI. Check for mistakes.
Comment on lines +8 to +12
const testDb = process.env.TYPEORM_DATABASE;
const client = new Client({
host: process.env.TYPEORM_HOST,
port: parseInt(process.env.TYPEORM_PORT, 10),
user: process.env.TYPEORM_USERNAME,
Copy link

Copilot AI Mar 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TYPEORM_DATABASE is later interpolated into DROP DATABASE IF EXISTS "${testDb}" without validation. Add the same safety checks as in setup (defined, not postgres, safe identifier) before running termination/drop queries so a misconfigured env can’t drop an unintended database.

Copilot uses AI. Check for mistakes.
Comment on lines +3 to +6
module.exports = {
...base,
testMatch: ['**/test/integration/**/*.test.[jt]s?(x)'],
};
Copy link

Copilot AI Mar 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If globalSetup/globalTeardown are moved out of the base Jest config to avoid impacting unit tests, add them here in the integration config so only integration runs create/drop the test DB.

Copilot uses AI. Check for mistakes.
module.exports = async function globalSetup() {
dotenv.config({ path: path.join(process.cwd(), '.env.test') });

const testDb = process.env.TYPEORM_DATABASE;
Copy link

Copilot AI Mar 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the current repo .env.test, TYPEORM_DATABASE is set to postgres, so this setup won’t create a separate test DB and teardown will attempt to drop postgres (which will error and is unsafe). Consider failing fast here when TYPEORM_DATABASE is postgres and instructing developers/CI to use a dedicated test DB name (e.g. upgrade_test).

Suggested change
const testDb = process.env.TYPEORM_DATABASE;
const testDb = process.env.TYPEORM_DATABASE;
if (!testDb) {
throw new Error(
'TYPEORM_DATABASE is not set in .env.test. Please configure a dedicated test database name, e.g. "upgrade_test".',
);
}
if (testDb === 'postgres') {
throw new Error(
'Refusing to use "postgres" as the test database. Please set TYPEORM_DATABASE in .env.test to a dedicated test database name, e.g. "upgrade_test".',
);
}

Copilot uses AI. Check for mistakes.
Comment on lines +56 to +59
globalSetup: './test/integration/setup/globalSetup.js',
setupFilesAfterEnv: ['./test/unit/lib/setup.ts'],
// A path to a module which exports an async function that is triggered once after all test suites
// globalTeardown: undefined,
globalTeardown: './test/integration/setup/globalTeardown.js',
Copy link

Copilot AI Mar 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

globalSetup/globalTeardown are now configured in the base Jest config, which means they will run for unit tests and any Jest invocation that uses jest.config.js (e.g., nps test.unit.run, nps test.coverage). This introduces an unexpected Postgres dependency for unit tests and can create/drop the test database even when running only unit tests. Move these hooks into jest.integration.config.js (or a dedicated integration-only config) and keep the base/unit config DB-free.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants