Site ↗
Documentation sections
Operations · 0.8.1

Migrations: changing database schema and data

Apply generated SQL, add your own migration, and investigate migration journal errors.

On this page

A migration is an SQL file that changes the database structure or its data. Generate creates these files but does not execute them in the database. The migrator applies them, started through make migrate, make dev, or the deployment controller (deploy/control).

Apply model changes to the dev database

  1. In Generator UI, change an entity, fields, relationships, or settings that affect the schema.
  2. Open Generate, review changes, and run generation.
  3. Review the new migration in backend/migrations before applying it to important data.
  4. From the dev project root, run:
make up
make migrate

make dev also applies pending migrations before starting the application. Rerunning does not execute already applied SQL again.

Files involved

File or table Purpose
backend/migrations/manifest.json Shared sequence of system, generated, and custom migrations
NNNNNN_system_*.sql System schema, such as users and media
NNNNNN_generated_*.sql Changes derived from the model
custom/NNNNNN_custom_*.sql Your SQL changes
admingen_internal.schema_migrations Journal of executed steps and checksums in PostgreSQL

The number determines execution order. Each SQL file must be registered in manifest.json: the migrator does not apply a file merely because it is in the folder.

Add your own SQL migration

After the first Generate, from the project root:

make migration-new NAME=backfill_labels

Equivalent CLI command:

admingen-cli migration create --project . --name backfill_labels

The command creates a file in backend/migrations/custom and registers it with the next available number in manifest.json. The database does not change at this step. Open the file at the path printed by the command, write SQL, and commit both the SQL file and the changed manifest to Git.

Example of backfilling a new nullable field in existing records:

UPDATE "articles"
SET "label" = "title"
WHERE "label" IS NULL;

Use your project's actual table and fields. Then apply the migration with make migrate. On the next Generate, the generator continues the shared sequence and preserves custom SQL.

Check connectivity and applied migrations

From the dev project root:

make wait-db
make wait-db WAIT_DB_CMD='cd backend && APP_PROFILE=local go run ./cmd/migrate --verify'

The first command checks connectivity using --check. The second substitutes --verify to check the migration journal as well. Both obtain connection settings from the project environment through Makefile and do not change the database.

--check checks only the PostgreSQL connection. --verify checks that the expected migration journal is fully applied and consistent with the files; it executes no SQL. Success means exit code 0.

In a running backend container:

docker exec ИМЯ_КОНТЕЙНЕРА_BACKEND migrate --verify

In production

For a server update, include migrations in the new release and run deploy. The controller temporarily closes application access, creates a backup, applies the release's SQL, and checks readiness. Only then does the release become current. Do not apply SQL from a local folder to a running server separately from its release.

The default migrator timeout is 30 minutes. Set another duration with MIGRATION_TIMEOUT using Go duration syntax: for example, 45m means 45 minutes. Before increasing the limit, find which query is delaying the migration.

Errors and fixes

Error Action
PostgreSQL unavailable Check make up, address, port, and connection settings
Applied migration checksum changed Restore the original SQL from Git; put the fix in a new migration
Journal incomplete or inconsistent with the manifest Compare code version, SQL files, and dump source; do not edit the journal manually
New migration SQL failed Read the cause, fix SQL that has not yet been applied, and retry; previous successful steps remain applied

SQL from one file and its application record are saved in a single transaction. Once successfully applied in any environment, the file must no longer be edited. Otherwise one migration number would mean different SQL on different servers. Make corrections through new migrations.

Rolling back an image does not undo SQL or restore deleted data. Before a potentially destructive change, create a backup and plan compatibility between old and new code. Restore data from a consistent backup point.

Checking foreign-key constraints

If a constraint was added as NOT VALID, PostgreSQL already checks new changes, but existing rows still need a separate check. List constraints and validate a selected one from the dev project root:

make constraints-list
make constraint-validate NAME=fk_users_role_id__roles_id

Take the name from the constraints-list result rather than the example. For all pending constraints:

make constraints-validate

If validation finds rows referencing missing records, fix that data and retry. Do not delete the constraint to hide the error. These commands use the dev connection from Makefile. On a server, perform validation separately, selecting the correct code version and database connection beforehand.