use set up and tear down test db during backend tests#3038
use set up and tear down test db during backend tests#3038danoswaltCL wants to merge 1 commit intodevfrom
Conversation
…t emails and users
There was a problem hiding this comment.
Pull request overview
Adds integration-test DB lifecycle management so backend integration tests can run against an isolated Postgres database (instead of clobbering local dev data), and removes hardcoded personal emails/users from test fixtures/config.
Changes:
- Add Jest global setup/teardown scripts to create/drop the integration test database.
- Introduce a dedicated Jest integration config and update integration test scripts to use it.
- Remove hardcoded user/email references in integration test data and delete
src/config.json.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/backend/test/integration/setup/globalSetup.js | Creates the test database before integration suites run. |
| packages/backend/test/integration/setup/globalTeardown.js | Terminates connections and drops the test database after integration suites run. |
| packages/backend/jest.integration.config.js | New integration-only Jest config (currently only narrows test matching). |
| packages/backend/jest.config.js | Enables Jest global setup/teardown hooks (currently applied globally). |
| packages/backend/package-scripts.js | Updates integration test commands to use jest.integration.config.js. |
| packages/backend/test/integration/mockData/user/index.ts | Replaces hardcoded personal email/name with mock values. |
| packages/backend/test/integration/Experiment/analytics/MonitoredPointForExport.ts | Uses systemUser.email instead of a hardcoded email string. |
| packages/backend/src/config.json | Removes a backend config file containing hardcoded user list/secrets. |
| packages/backend/.gitignore | Un-ignores integration setup JS files so they can be committed. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| const path = require('path'); | ||
|
|
||
| module.exports = async function globalSetup() { | ||
| dotenv.config({ path: path.join(process.cwd(), '.env.test') }); |
There was a problem hiding this comment.
Using dotenv.config({ path: path.join(process.cwd(), '.env.test') }) makes setup dependent on the process working directory (e.g., running Jest from the repo root with --config packages/backend/jest.integration.config.js won’t find the env file). Prefer resolving relative to this file (via __dirname) or Jest’s <rootDir> so the env file path is stable across local/CI execution contexts.
| dotenv.config({ path: path.join(process.cwd(), '.env.test') }); | |
| dotenv.config({ path: path.join(__dirname, '../../../.env.test') }); |
| module.exports = async function globalSetup() { | ||
| dotenv.config({ path: path.join(process.cwd(), '.env.test') }); | ||
|
|
||
| const testDb = process.env.TYPEORM_DATABASE; |
There was a problem hiding this comment.
TYPEORM_DATABASE is used to determine which database to create, but it isn’t validated. Add guards to ensure it’s defined, not postgres, and matches a safe identifier pattern (e.g. /^[A-Za-z0-9_]+$/) before it’s later interpolated into CREATE DATABASE "${testDb}", to avoid targeting/dropping the wrong DB and to prevent identifier-injection via env vars.
| const testDb = process.env.TYPEORM_DATABASE; | |
| const testDb = process.env.TYPEORM_DATABASE; | |
| const dbNamePattern = /^[A-Za-z0-9_]+$/; | |
| if (!testDb) { | |
| throw new Error('TYPEORM_DATABASE environment variable must be defined for test setup'); | |
| } | |
| if (testDb.toLowerCase() === 'postgres') { | |
| throw new Error('TYPEORM_DATABASE must not be set to "postgres" for test setup'); | |
| } | |
| if (!dbNamePattern.test(testDb)) { | |
| throw new Error('TYPEORM_DATABASE contains invalid characters; only A-Za-z0-9_ are allowed'); | |
| } |
| const result = await client.query('SELECT 1 FROM pg_database WHERE datname = $1', [testDb]); | ||
| if (result.rowCount === 0) { | ||
| await client.query(`CREATE DATABASE "${testDb}"`); | ||
| console.log(`Created test database: ${testDb}`); | ||
| } |
There was a problem hiding this comment.
If a previous test run exits before globalTeardown (e.g., SIGKILL/CI timeout), the test DB will remain and globalSetup will currently reuse it without resetting state (it only creates if missing). To guarantee a clean run, consider dropping/recreating the DB in setup (after terminating connections) or otherwise clearing schema/data deterministically.
| const result = await client.query('SELECT 1 FROM pg_database WHERE datname = $1', [testDb]); | |
| if (result.rowCount === 0) { | |
| await client.query(`CREATE DATABASE "${testDb}"`); | |
| console.log(`Created test database: ${testDb}`); | |
| } | |
| // Ensure no active connections to the test database from previous runs | |
| await client.query( | |
| 'SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE datname = $1 AND pid <> pg_backend_pid()', | |
| [testDb], | |
| ); | |
| // Drop and recreate the test database to guarantee a clean state | |
| await client.query(`DROP DATABASE IF EXISTS "${testDb}"`); | |
| await client.query(`CREATE DATABASE "${testDb}"`); | |
| console.log(`Recreated test database: ${testDb}`); |
|
|
||
| module.exports = async function globalTeardown() { | ||
| dotenv.config({ path: path.join(process.cwd(), '.env.test') }); |
There was a problem hiding this comment.
Like setup, teardown relies on process.cwd() to locate .env.test, which can break when Jest is invoked from a different working directory. Resolve the env file path relative to the repo/workspace instead of the current process CWD to make teardown reliable in CI and when running from the monorepo root.
| module.exports = async function globalTeardown() { | |
| dotenv.config({ path: path.join(process.cwd(), '.env.test') }); | |
| const fs = require('fs'); | |
| function findEnvTest(startDir) { | |
| let dir = startDir; | |
| while (true) { | |
| const candidate = path.join(dir, '.env.test'); | |
| if (fs.existsSync(candidate)) { | |
| return candidate; | |
| } | |
| const parent = path.dirname(dir); | |
| if (parent === dir) { | |
| break; | |
| } | |
| dir = parent; | |
| } | |
| return null; | |
| } | |
| module.exports = async function globalTeardown() { | |
| const envPath = findEnvTest(__dirname); | |
| if (envPath) { | |
| dotenv.config({ path: envPath }); | |
| } else { | |
| dotenv.config(); | |
| } |
| const testDb = process.env.TYPEORM_DATABASE; | ||
| const client = new Client({ | ||
| host: process.env.TYPEORM_HOST, | ||
| port: parseInt(process.env.TYPEORM_PORT, 10), | ||
| user: process.env.TYPEORM_USERNAME, |
There was a problem hiding this comment.
TYPEORM_DATABASE is later interpolated into DROP DATABASE IF EXISTS "${testDb}" without validation. Add the same safety checks as in setup (defined, not postgres, safe identifier) before running termination/drop queries so a misconfigured env can’t drop an unintended database.
| module.exports = { | ||
| ...base, | ||
| testMatch: ['**/test/integration/**/*.test.[jt]s?(x)'], | ||
| }; |
There was a problem hiding this comment.
If globalSetup/globalTeardown are moved out of the base Jest config to avoid impacting unit tests, add them here in the integration config so only integration runs create/drop the test DB.
| module.exports = async function globalSetup() { | ||
| dotenv.config({ path: path.join(process.cwd(), '.env.test') }); | ||
|
|
||
| const testDb = process.env.TYPEORM_DATABASE; |
There was a problem hiding this comment.
With the current repo .env.test, TYPEORM_DATABASE is set to postgres, so this setup won’t create a separate test DB and teardown will attempt to drop postgres (which will error and is unsafe). Consider failing fast here when TYPEORM_DATABASE is postgres and instructing developers/CI to use a dedicated test DB name (e.g. upgrade_test).
| const testDb = process.env.TYPEORM_DATABASE; | |
| const testDb = process.env.TYPEORM_DATABASE; | |
| if (!testDb) { | |
| throw new Error( | |
| 'TYPEORM_DATABASE is not set in .env.test. Please configure a dedicated test database name, e.g. "upgrade_test".', | |
| ); | |
| } | |
| if (testDb === 'postgres') { | |
| throw new Error( | |
| 'Refusing to use "postgres" as the test database. Please set TYPEORM_DATABASE in .env.test to a dedicated test database name, e.g. "upgrade_test".', | |
| ); | |
| } |
| globalSetup: './test/integration/setup/globalSetup.js', | ||
| setupFilesAfterEnv: ['./test/unit/lib/setup.ts'], | ||
| // A path to a module which exports an async function that is triggered once after all test suites | ||
| // globalTeardown: undefined, | ||
| globalTeardown: './test/integration/setup/globalTeardown.js', |
There was a problem hiding this comment.
globalSetup/globalTeardown are now configured in the base Jest config, which means they will run for unit tests and any Jest invocation that uses jest.config.js (e.g., nps test.unit.run, nps test.coverage). This introduces an unexpected Postgres dependency for unit tests and can create/drop the test database even when running only unit tests. Move these hooks into jest.integration.config.js (or a dedicated integration-only config) and keep the base/unit config DB-free.
this will create a separate test db when running backend tests, as it currently uses the same as local dev and clobbers that data without cleanup, we don't have to live this way
also cleans out some unneeded harcoded users and vivek email address
hopefully this works on jenkins, I guess we'll see. if not, will see where to conditionally make sure cicd still uses
postgresfor the tests.