Integrating an app with an ERP that has no idempotency
Context
A multi-warehouse inventory web app had to record every stock movement (reservation, picking, return) in the client's TOTVS Datasul ERP through a REST API. Each user action in the app becomes a call to the ERP.
The problem
Three things made the integration risky:
- The ERP has no idempotency. If a call times out, I can't tell whether it went through. Retrying may duplicate the movement, and not retrying may lose it.
- The HTTP status doesn't tell the whole story. Business errors arrive as HTTP 200 with an
errorfield in the body. - Incomplete data passes in the app and breaks in the ERP, such as an item without a numeric code, zero quantity or a shipment without a project.
What I decided
1. Classify every response into 4 outcomes
I look at the HTTP status first and then at the body:
confirmado(confirmed): 2xx with no error in the body.rejeitado(rejected): business error. No retry, and the reservation is rolled back.falha(failure): infrastructure problem (401, 403, 404, 429, 5xx, connection refused). This is the only case that retries, with exponential backoff, ±20% jitter and honoringRetry-After.divergente(diverged): uncertain outcome.
2. When in doubt, stop and call a person
Timeouts, unreadable bodies and redirects go to divergente and are never retried automatically, because the ERP may have applied them. Only a human resends. I prefer a visible, stalled movement over a silent duplicate.
3. A unique mark on every movement
The notes field always carries a movement identifier. If something duplicates or stays uncertain, it can be located in the ERP by that mark and reconciled.
4. Validate before calling the API
The translation from the app to the ERP contract is a pure layer with no I/O. If the data is invalid (no project, item without a numeric code, quantity ≤ 0), the movement becomes divergente without even calling the ERP.
5. A queue that protects itself
- Sends in batches of 20, with a lease so two processes don't take the same batch.
- After 3 consecutive infrastructure failures, the cycle pauses for 60 s instead of burning attempts of movements that aren't at fault.
- At most 10 attempts; beyond that the movement leaves the queue and goes to human intervention.
- Configurable concurrency (1 to 10, default 1), so it starts slowly.
6. Security and operating modes
Credentials only go out over HTTPS (or localhost), and redirects are not followed. The app runs in three modes: simulado (simulated), api and desligado (off). An invalid configuration value falls back to desligado, never simulado, so a config mistake can't look like success.
Result
- Everything about the ERP (field names, codes, item format) is isolated in a single pure file. If the ERP changes, I touch one place.
- That contract has 26 automated tests (validations, limits, reading responses with invalid input). I broke a rule on purpose to check that a test fails.
- The integration is in a staging environment, not in production yet.
What I learned
- When the system on the other side offers no guarantee, the guarantee becomes my responsibility, and the design has to accept uncertainty instead of hiding it.
- Being explicit about what is an assumption (I marked in the code what I had inferred from examples) avoids treating a guess as a contract.
- Contract tests are where it pays off most to start when a project had none.