This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Documentation language
**Toda la documentación del proyecto debe estar en español**: comentarios de código, README, docstrings y cualquier texto explicativo nuevo o modificado. Los identificadores de código (nombres de funciones, variables, tipos) siguen en inglés; solo la prosa explicativa va en español.
## What this service does
## What this service does
Runs two independent full-sync pipelines every `SYNC_INTERVAL`, full-syncing the result set into a MongoDB collection: students (raw SQL query joining `matricula.ma_estudiante`/`persona.pe_persona`/`matricula.ma_matricula`/`caja.ca_plan_de_pago`, upsert by `student_id`, collection `students`) and parents (Postgres function `func_apoderado_listar`, upsert by `parent_id`, collection `parents`). Sincroniza estudiantes (con padres y planes de pago) y apoderados desde PostgreSQL hacia MongoDB. No incremental/CDC logic — every cycle re-reads the full dataset for that pipeline and reconciles Mongo against it (upsert what's present, archive what's gone).
Runs one full-sync pipeline every `SYNC_INTERVAL`, full-syncing the result set into a MongoDB collection: students (raw SQL query joining `matricula.ma_estudiante`/`persona.pe_persona`/`matricula.ma_matricula`/`caja.ca_plan_de_pago`, upsert by `student_id`, collection `students`). Sincroniza estudiantes (con padres y planes de pago) desde PostgreSQL hacia MongoDB. No incremental/CDC logic — every cycle re-reads the full dataset and reconciles Mongo against it (upsert what's present, archive what's gone).
## Commands
## Commands
...
@@ -18,46 +22,42 @@ No Makefile/linter config present, no test suite — use plain `go` toolchain co
...
@@ -18,46 +22,42 @@ No Makefile/linter config present, no test suite — use plain `go` toolchain co
## Setup / running locally
## Setup / running locally
1.`sync_state` table is auto-bootstrapped at startup (`Pool.BootstrapSchema`, seeds `id=1` students / `id=2` parents rows) — no manual migration step required, though the SQL files under `migrations/` are kept for reference/manual runs.
1.`sync_state` table is auto-bootstrapped at startup (`Pool.BootstrapSchema`, seeds `id=1` students row) — no manual migration step required, though the SQL files under `migrations/` are kept for reference/manual runs.
2. Mongo `students` collection must exist with a `$jsonSchema` validator requiring `student_id` int and `enrollment_id` int. A `deleted_students` collection archives students removed at the source — no schema setup needed for it.
2. Mongo `students` collection must exist with a `$jsonSchema` validator requiring `student_id` int and `enrollment_id` int. A `deleted_students` collection archives students removed at the source — no schema setup needed for it.
3. Mongo `parents` collection must exist with a `$jsonSchema` validator requiring `parent_id` int. A `deleted_parents` collection archives parents removed at the source — no schema setup needed for it.
3. Required env vars: `PG_DSN`, `MONGO_URI`, `MONGO_DB`. Optional: `SYNC_INTERVAL` (default `5m`), `PORT` (default `8080`). Per-pipeline PG function name (or raw `PGQuery`) / Mongo collection / deleted-collection / id field / `sync_state` id are **not** env vars — they're declared in `internal/config/pipelines.go` (`Pipelines []PipelineDef`). Add a new sync pipeline by adding an entry there; `cmd/server/main.go` loops over `config.Pipelines` to wire reader/upserter/mover/state/service/scheduler for each.
4. Required env vars: `PG_DSN`, `MONGO_URI`, `MONGO_DB`. Optional: `SYNC_INTERVAL` (default `5m`, shared by all pipelines), `PORT` (default `8080`). Per-pipeline PG function name (or raw `PGQuery`) / Mongo collection / deleted-collection / id field / `sync_state` id are **not** env vars — they're declared in `internal/config/pipelines.go` (`Pipelines []PipelineDef`). Add a new sync pipeline by adding an entry there; `cmd/server/main.go` loops over `config.Pipelines` to wire reader/upserter/mover/state/service/scheduler for each.
4.`go run ./cmd/server`
5.`go run ./cmd/server`
## Endpoints
## Endpoints
-`GET /health` — checks Postgres and Mongo connectivity.
-`GET /health` — checks Postgres and Mongo connectivity.
-`POST /sync/trigger` / `GET /sync/status` — students pipeline: run now / last run time, rows synced (created/updated/deleted breakdown), last error if any.
-`POST /sync/students/trigger` / `GET /sync/students/status` — students pipeline: run now / last run time, rows synced (created/updated/deleted breakdown), last error if any.
-`POST /sync/parents/trigger` / `GET /sync/parents/status` — same, for the parents pipeline.
## Postgres source contract
## Postgres source contract
Students: `internal/config.studentsQuery`, a raw SQL query (not a function) joining `matricula.ma_estudiante`/`persona.pe_persona`/`matricula.ma_matricula`/`caja.ca_plan_de_pago`, grouped per student, `payment_plans`/`payment_plans_current_year` built via `JSON_AGG(...) FILTER (...)` split on `periodo_academico_id = 14`. Read directly by `db.StudentsQueryReader` (`pgx` row scan, no JSON envelope) — one Mongo doc per row, upserted by `_id = student_id`, requires numeric `enrollment_id`.
Students: `internal/config.studentsQuery`, a raw SQL query (not a function) joining `matricula.ma_estudiante`/`persona.pe_persona`/`matricula.ma_matricula`/`caja.ca_plan_de_pago`, grouped per student, `payment_plans`/`payment_plans_current_year` built via `JSON_AGG(...) FILTER (...)` split on `periodo_academico_id = 14`. Read directly by `db.StudentsQueryReader` (`pgx` row scan, no JSON envelope) — one Mongo doc per row, upserted by `_id = student_id`, requires numeric `enrollment_id`.
Parents: `PG_APODERADOS_FUNCTION` (`func_apoderado_listar()`), takes no params, returns one JSON envelope: `{"status": bool, "message": text, "data": [...]}`, unwrapped by `db.GenericFunctionReader`. Each `data[]` item → one Mongo doc, upserted by `_id = parent_id`.
`updated_at` is stamped by this service (Postgres does not supply it).
`updated_at` is stamped by this service for both pipelines (Postgres does not supply it).
## Sync behavior
## Sync behavior
Each cycle (students and parents run independently) processes rows concurrently via a bounded worker pool (`defaultConcurrency` = 10, `errgroup`-based) so a full migration finishes faster than a sequential loop:
Each cycle processes rows concurrently via a bounded worker pool (`defaultConcurrency` = 10, `errgroup`-based) so a full migration finishes faster than a sequential loop:
-**New record**: id not yet in Mongo → inserted, counted as `Created`.
-**New record**: id not yet in Mongo → inserted, counted as `Created`.
-**Existing record**: id already in Mongo → fields overwritten, counted as `Updated`.
-**Existing record**: id already in Mongo → fields overwritten, counted as `Updated`.
-**Removed record**: id present in Mongo but no longer returned by the Postgres source (query or function) → moved (not just deleted) into the pipeline's archive collection (`deleted_students` / `deleted_parents`) with a `deleted_at` stamp, counted as `Deleted`.
-**Removed record**: id present in Mongo but no longer returned by the Postgres source → moved (not just deleted) into the archive collection (`deleted_students`) with a `deleted_at` stamp, counted as `Deleted`.
Persisted `sync_state` only advances if the entire cycle (upserts + delete reconciliation) succeeds; any failure short-circuits and is surfaced via `/sync/status`.
Persisted `sync_state` only advances if the entire cycle (upserts + delete reconciliation) succeeds; any failure short-circuits and is surfaced via `/sync/students/status`.
## Architecture
## Architecture
Layering, dependency direction is `api` / `sync` → small interfaces, with concrete Postgres/Mongo clients living in `internal/db` and injected from `cmd/server/main.go`:
Layering, dependency direction is `api` / `sync` → small interfaces, with concrete Postgres/Mongo clients living in `internal/db` and injected from `cmd/server/main.go`:
-`internal/config` — `config.go`: env var loading/validation (`config.Load()`, only `PGDSN`/`MongoURI`/`MongoDB`/`SyncInterval`/`Port`), fails fast on missing required vars. `pipelines.go`: `Pipelines []PipelineDef` — static Go-defined list of sync pipelines (PG function or raw `PGQuery`, Mongo collection, deleted collection, id field, `sync_state` id, `RequireEnrollmentID` flag). Adding a pipeline means adding an entry here, not new env vars. Also holds `studentsQuery`, the raw SQL for the students pipeline.
-`internal/config` — `config.go`: env var loading/validation (`config.Load()`, only `PGDSN`/`MongoURI`/`MongoDB`/`SyncInterval`/`Port`), fails fast on missing required vars. `pipelines.go`: `Pipelines []PipelineDef` — static Go-defined list of sync pipelines (PG function or raw `PGQuery`, Mongo collection, deleted collection, id field, `sync_state` id, `RequireEnrollmentID` flag). Adding a pipeline means adding an entry here, not new env vars. Also holds `studentsQuery`, the raw SQL for the students pipeline.
-`internal/db` — `NewPostgresPool`, `NewMongoClient`, `StudentsQueryReader` (students-specific: runs `PGQuery` directly via `pgx` row scan, no JSON envelope, requires `student_id`+`enrollment_id`), `FunctionReader` (legacy: calls a PG function returning a JSON envelope, unwraps into `[]sync.SyncRow`, requires `student_id`+`enrollment_id` — kept for pipelines not yet migrated to raw queries), `GenericFunctionReader` (parents-and-beyond: same envelope unwrap, validates only a configurable id field), `CollectionUpserter` (Mongo upsert by int id, also implements `ListIDs` for delete reconciliation), `DeletedMover` (moves a doc from a source collection into an archive collection, stamping `deleted_at`).
-`internal/db` — `NewPostgresPool`, `NewMongoClient`, `StudentsQueryReader` (students-specific: runs `PGQuery` directly via `pgx` row scan, no JSON envelope, requires `student_id`+`enrollment_id`), `FunctionReader` (legacy: calls a PG function returning a JSON envelope, unwraps into `[]sync.SyncRow`, requires `student_id`+`enrollment_id` — kept for pipelines not yet migrated to raw queries), `GenericFunctionReader` (generic: same envelope unwrap, validates only a configurable id field), `CollectionUpserter` (Mongo upsert by int id, also implements `ListIDs` for delete reconciliation), `DeletedMover` (moves a doc from a source collection into an archive collection, stamping `deleted_at`).
-`internal/sync` — core domain, framework-free:
-`internal/sync` — core domain, framework-free:
-`service.go`: `Service.Run(ctx)` does one full cycle — fetch all rows, concurrently upsert each with `updated_at` stamped (tracking created vs. updated counts via `MongoUpserter`), then reconcile deletions by diffing Mongo's existing ids (`MongoIDLister`) against the current Postgres id set and archiving the difference (`DeletedMover`). Advances persisted state only on full success; stops and records the error in `lastResult` otherwise. `LastResult()`/`SeedLastResult()` back the `/sync/status` endpoint and boot-time state hydration.
-`service.go`: `Service.Run(ctx)` does one full cycle — fetch all rows, concurrently upsert each with `updated_at` stamped (tracking created vs. updated counts via `MongoUpserter`), then reconcile deletions by diffing Mongo's existing ids (`MongoIDLister`) against the current Postgres id set and archiving the difference (`DeletedMover`). Advances persisted state only on full success; stops and records the error in `lastResult` otherwise. `LastResult()`/`SeedLastResult()` back the `/sync/status` endpoint and boot-time state hydration.
-`scheduler.go`: `Scheduler.Start(ctx)` ticks every `interval` and calls `Run` with a hard `cycleTimeout` (4 min) sub-context per cycle, independent of the ticker interval.
-`scheduler.go`: `Scheduler.Start(ctx)` ticks every `interval` and calls `Run` with a hard `cycleTimeout` (4 min) sub-context per cycle, independent of the ticker interval.
-`state.go`: `StateStore` interface + `PGStateStore` (constructed with an explicit `stateID`: students=1, parents=2), persists last-synced timestamp in the `sync_state` table so `/sync/status`, `/sync/parents/status`, and boot seeding survive restarts for each pipeline.
-`state.go`: `StateStore` interface + `PGStateStore` (constructed with an explicit `stateID`: students=1), persists last-synced timestamp in the `sync_state` table so `/sync/students/status` and boot seeding survive restarts.
- Depends only on small interfaces (`PGReader`, `MongoUpserter`, `MongoIDLister`, `DeletedMover`, `StateStore`) — no real DB/Mongo needed to exercise it.
- Depends only on small interfaces (`PGReader`, `MongoUpserter`, `MongoIDLister`, `DeletedMover`, `StateStore`) — no real DB/Mongo needed to exercise it.
-`cmd/server/main.go` — composition root: builds config → DB clients → two independent reader/upserter/mover/state/`Service`/`Scheduler` sets (students, parents) → seeds each `Service`'s last result from its persisted state → starts both `Scheduler`s in goroutines → starts HTTP server → graceful shutdown on SIGINT/SIGTERM (10s timeout). The two pipelines share nothing at runtime beyond the Postgres pool and Mongo client — separate scheduler tick, separate `sync_state` row, separate failure domain.
-`cmd/server/main.go` — composition root: builds config → DB clients → per-pipeline reader/upserter/mover/state/`Service`/`Scheduler` set (looping over `config.Pipelines`, currently just students) → seeds each `Service`'s last result from its persisted state → starts each `Scheduler` in a goroutine → starts HTTP server → graceful shutdown on SIGINT/SIGTERM (10s timeout).
Sincroniza estudiantes (con padres y planes de pago) y apoderados desde PostgreSQL hacia MongoDB mediante sondeo completo cada N minutos, en dos pipelines independientes.
Sincroniza estudiantes (con padres y planes de pago) desde PostgreSQL hacia MongoDB mediante sondeo completo cada N minutos.
## Configuración
## Configuración
1. La tabla `sync_state` se crea/inicializa automáticamente al arrancar (filas `id=1` estudiantes, `id=2` apoderados) — no se requiere paso de migración manual (`migrations/0001_create_sync_state.sql` se conserva como referencia).
1. La tabla `sync_state` se crea/inicializa automáticamente al arrancar (fila`id=1` estudiantes) — no se requiere paso de migración manual (`migrations/0001_create_sync_state.sql` se conserva como referencia).
2. Asegúrate de que la colección `students` exista en Mongo con el validador `$jsonSchema` provisto (requeridos: `student_id` int, `enrollment_id` int). La colección `deleted_students` archiva los estudiantes eliminados en el origen — no requiere configuración de esquema.
2. Asegúrate de que la colección `students` exista en Mongo con el validador `$jsonSchema` provisto (requeridos: `student_id` int, `enrollment_id` int). La colección `deleted_students` archiva los estudiantes eliminados en el origen — no requiere configuración de esquema.
3. Asegúrate de que la colección `parents` exista en Mongo con un validador `$jsonSchema` que requiera `parent_id` int. La colección `deleted_parents` archiva los apoderados eliminados en el origen — no requiere configuración de esquema.
3. Define las variables de entorno: `PG_DSN`, `MONGO_URI`, `MONGO_DB=intranet` (o tu base de datos), `SYNC_INTERVAL` (por defecto `5m`), `PORT` (por defecto `8080`). Los nombres de función PG / colección Mongo por pipeline ya no son variables de entorno — se definen en `internal/config/pipelines.go`. Agrega un nuevo pipeline añadiendo una entrada ahí, no variables de entorno.
4. Define las variables de entorno: `PG_DSN`, `MONGO_URI`, `MONGO_DB=intranet` (o tu base de datos), `SYNC_INTERVAL` (por defecto `5m`, compartido por todos los pipelines), `PORT` (por defecto `8080`). Los nombres de función PG / colección Mongo por pipeline ya no son variables de entorno — se definen en `internal/config/pipelines.go`. Agrega un nuevo pipeline añadiendo una entrada ahí, no variables de entorno.
4.`go run ./cmd/server`
5.`go run ./cmd/server`
## Endpoints
## Endpoints
-`GET /health` — verifica la conectividad con Postgres y Mongo.
-`GET /health` — verifica la conectividad con Postgres y Mongo.
-`POST /sync/trigger` — ejecuta un ciclo de sincronización de estudiantes de inmediato.
-`POST /sync/students/trigger` — ejecuta un ciclo de sincronización de estudiantes de inmediato.
-`GET /sync/status` — última ejecución de sincronización de estudiantes, filas sincronizadas (desglose creado/actualizado/eliminado), último error si lo hay.
-`GET /sync/students/status` — última ejecución de sincronización de estudiantes, filas sincronizadas (desglose creado/actualizado/eliminado), último error si lo hay.
-`POST /sync/parents/trigger` — ejecuta un ciclo de sincronización de apoderados de inmediato.
-`GET /sync/parents/status` — última ejecución de sincronización de apoderados, filas sincronizadas (desglose creado/actualizado/eliminado), último error si lo hay.
## Contratos del origen Postgres
## Contratos del origen Postgres
Estudiantes: una consulta SQL cruda (no una función) que une `matricula.ma_estudiante`/`persona.pe_persona`/`matricula.ma_matricula`/`caja.ca_plan_de_pago`, agrupada por estudiante. Cada fila se convierte en un documento Mongo, upsert por `_id = student_id`, con `updated_at` estampado por este servicio. Requiere `student_id` y `enrollment_id` numéricos.
Estudiantes: una consulta SQL cruda (no una función) que une `matricula.ma_estudiante`/`persona.pe_persona`/`matricula.ma_matricula`/`caja.ca_plan_de_pago`, agrupada por estudiante. Cada fila se convierte en un documento Mongo, upsert por `_id = student_id`, con `updated_at` estampado por este servicio. Requiere `student_id` y `enrollment_id` numéricos.
`func_apoderado_listar()` no recibe parámetros y devuelve el conjunto completo como un único envoltorio JSON: `{"status": bool, "message": text, "data": [...]}`. Cada elemento de `data[]` se convierte en un documento Mongo en la colección `parents`, upsert por `_id = parent_id`, con `updated_at` estampado por este servicio. Solo se requiere/valida `parent_id`.
## Comportamiento de sincronización
## Comportamiento de sincronización
Cada ciclo (estudiantes y apoderados de forma independiente) corre en concurrencia (pool de workers acotado, 10 en vuelo por defecto) para que una migración completa termine más rápido que un bucle secuencial:
Cada ciclo corre en concurrencia (pool de workers acotado, 10 en vuelo por defecto) para que una migración completa termine más rápido que un bucle secuencial:
-**Registro nuevo**: id aún no en Mongo → insertado, contado como `Created`.
-**Registro nuevo**: id aún no en Mongo → insertado, contado como `Created`.
-**Registro existente**: id ya en Mongo → campos sobrescritos, contado como `Updated`.
-**Registro existente**: id ya en Mongo → campos sobrescritos, contado como `Updated`.
-**Registro eliminado**: id presente en Mongo pero ya no devuelto por el origen Postgres → el documento se mueve (no solo se elimina) a la colección de archivo del pipeline (`deleted_students` / `deleted_parents`) con una marca `deleted_at`, contado como `Deleted`.
-**Registro eliminado**: id presente en Mongo pero ya no devuelto por el origen Postgres → el documento se mueve (no solo se elimina) a la colección de archivo (`deleted_students`) con una marca `deleted_at`, contado como `Deleted`.
Los dos pipelines son totalmente independientes: tick de scheduler separado, fila `sync_state` separada, dominio de fallos separado — un error del lado de estudiantes no bloquea la sincronización de apoderados ni viceversa.
El estado persistido (`sync_state`) solo avanza si el ciclo completo (upserts + reconciliación de borrados) tiene éxito; cualquier fallo se corta y se reporta vía `/sync/students/status`.