SAT bulk-download in production: a pipeline to pull CFDI for many RFCs (accounting firm), in Python
Model each SAT query as a durable job — one per (RFC, date window, tipo). Send SolicitaDescarga ONCE, persist the IdSolicitud, and poll VerificaSolicitud with backoff; never re-send an identical request (that is the 5005 duplicada → 5002 lifetime-cap trap). Download each package inside its 72h window, dedup by UUID, and sync incrementally — request only windows past your per-RFC watermark.
The short answer: for a despacho with many RFCs, model each SAT query as a durable job
To bulk-download CFDI for a despacho with many RFCs, model each SAT query as a durable job — one per (RFC, date-window, tipo) — instead of a function call buried in a for loop. Send SolicitaDescarga exactly once, persist the returned IdSolicitud, then poll VerificaSolicitud with exponential backoff until EstadoSolicitud = 3 (Terminada) and download the packages inside their ~72h window.
Never re-send an identical SolicitaDescarga — that is the 5005 duplicada → 5002 lifetime-cap trap. Dedup stored CFDI by UUID and sync incrementally: request only windows past each RFC’s watermark, never old ranges again.
I am an engineer who ships this in production, not your contador or abogado. SAT rules and codes change; verify the fiscal specifics against sat.gob.mx and the canonical phpcfdi/sat-ws-descarga-masiva. What follows is the systems layer.
The despacho reality: N clients × emitidas/recibidas × Metadata/CFDI
A single-RFC tutorial hides the real problem. A despacho does not have one taxpayer — it has dozens, each needing both issued (RfcEmisor) and received (RfcReceptor) invoices, and for each you often want a light Metadata pass before pulling full CFDI XML. Multiply that by a 12-month backfill split into windows and one accounting firm becomes thousands of independent SAT requests, each with its own lifecycle and its own 72-hour clock.
You cannot hold that in memory inside a script that runs to completion. Any process that dies — a deploy, an OOM kill, a token expiry — must resume exactly where it left off without re-issuing a single request. That requirement alone forces the durable-job design.
Why the naive for-loop breaks: 5005 duplicada, then 5002 se agotaron las solicitudes de por vida
The single most common production failure is this: a developer sees SolicitaDescarga return CodEstatus 5000 — “Solicitud recibida con éxito” — assumes it means the data is ready, gets nothing back, and re-sends the same request. But 5000 is only the acknowledgment that your request was received. Readiness is reported separately by VerificaSolicitud through EstadoSolicitud.
Re-sending the identical query — same FechaInicial, FechaFinal, RfcEmisor, RfcReceptor, and TipoSolicitud — is exactly what the SAT rejects. First you get 5005 Solicitud duplicada (a request with those parameters already exists; reuse its IdSolicitud, do not create a new one). Keep hammering and you eventually hit 5002 Se agotaron las solicitudes de por vida — a lifetime cap on that exact query. At that point that specific window is burned. I cover the full code table in the status and error codes reference; the takeaway here is architectural.
Naive for-loop (breaks in prod)
- Re-sends SolicitaDescarga because it keeps returning 5000/aceptada
- Same params again → 5005 solicitud duplicada
- Keeps retrying → 5002 se agotaron las solicitudes de por vida
- Loses IdSolicitud on restart and starts over
- No 72h tracking → packages expire (Vencida) unnoticed
Durable job (survives prod)
- SolicitaDescarga sent ONCE, IdSolicitud persisted immediately
- Polls VerificaSolicitud with backoff on the saved IdSolicitud
- Idempotency key on (RFC, tipo, role, range) blocks duplicates
- Resumes mid-flight after any restart
- Marks Vencida at 72h and re-queues the window cleanly
Model each request as a durable job (a state machine you persist)
The fix is the same idempotency mindset I use for the timbrado outbox: a request is a row in a table, not a stack frame. Give it a natural idempotency key of (rfc, tipo, role, fecha_inicial, fecha_final) — which maps one-to-one onto the SAT’s own dedup parameters — and a unique constraint on that key. Now a duplicate SolicitaDescarga is impossible by construction: you look up the job first, and if it already has an IdSolicitud, you never call again.
from dataclasses import dataclass, field
from datetime import date, datetime, timezone
from enum import Enum
class JobState(str, Enum):
PENDING = "pending" # created; SolicitaDescarga not sent yet
REQUESTED = "requested" # SolicitaDescarga sent; IdSolicitud persisted
IN_PROGRESS = "in_progress" # VerificaSolicitud says Aceptada/EnProceso
FINISHED = "finished" # EstadoSolicitud=3 Terminada; packages listed
DOWNLOADED = "downloaded" # all packages pulled and stored
FAILED = "failed" # EstadoSolicitud in (4 Error, 5 Rechazada)
EXPIRED = "expired" # EstadoSolicitud=6 Vencida (past ~72h)
@dataclass
class DownloadJob:
rfc: str # the client (taxpayer) RFC
tipo: str # "CFDI" or "Metadata"
role: str # "emitidas" -> RfcEmisor, "recibidas" -> RfcReceptor
fecha_inicial: date
fecha_final: date
state: JobState = JobState.PENDING
id_solicitud: str | None = None # persisted after SolicitaDescarga returns 5000
estado_solicitud: int | None = None # last EstadoSolicitud from VerificaSolicitud
id_paquetes: list[str] = field(default_factory=list)
attempts: int = 0
requested_at: datetime | None = None
@property
def key(self) -> tuple:
return (self.rfc, self.tipo, self.role, self.fecha_inicial, self.fecha_final)Sending the request is guarded so it fires at most once, and it persists IdSolicitud before anything else can retry:
def submit_once(store, client, job: DownloadJob) -> DownloadJob:
if job.id_solicitud: # already requested -> never re-send
return job
r = client.solicita_descarga(
rfc_solicitante=job.rfc,
tipo_solicitud=job.tipo,
rfc_emisor=job.rfc if job.role == "emitidas" else None,
rfc_receptor=job.rfc if job.role == "recibidas" else None,
fecha_inicial=job.fecha_inicial,
fecha_final=job.fecha_final,
)
if r.cod_estatus in ("5000", "5005"): # 5000 accepted, 5005 duplicada -> reuse Id
job.id_solicitud = r.id_solicitud
job.state = JobState.REQUESTED
job.requested_at = datetime.now(timezone.utc)
store.upsert(job) # persist immediately, then return
return jobThe poller done right: VerificaSolicitud with backoff, and every EstadoSolicitud handled
The poller drives one job from REQUESTED to a terminal state. It reads EstadoSolicitud and branches on all six values — 1 Aceptada, 2 EnProceso, 3 Terminada, 4 Error, 5 Rechazada, 6 Vencida — backs off between polls, caps attempts, and treats the 72-hour boundary as Vencida. Only Terminada yields IdsPaquetes.
- 1 Aceptada / 2 EnProcesoKeep polling; sleep with exponential backoff, cap the interval at 5 min
- 3 TerminadaOnly here you get IdsPaquetes and NumeroCFDIs — advance to Descarga
- 4 Error / 5 RechazadaTerminal failure; log CodEstatus, alert, do not retry blindly
- 6 VencidaResult expired past ~72h; re-queue the window as a fresh job
import time
from datetime import datetime, timedelta, timezone
ACEPTADA, EN_PROCESO, TERMINADA, ERROR, RECHAZADA, VENCIDA = 1, 2, 3, 4, 5, 6
def poll_solicitud(store, client, job, *, max_attempts=18, base=30, cap=300):
"""Drive one job via VerificaSolicitud. Never re-sends SolicitaDescarga."""
deadline = job.requested_at + timedelta(hours=72) # packages live ~72h
for attempt in range(max_attempts):
if datetime.now(timezone.utc) > deadline:
job.state = JobState.EXPIRED
return store.upsert(job)
r = client.verifica_solicitud(job.id_solicitud) # one SOAP call
job.estado_solicitud = r.estado_solicitud
job.attempts += 1
if r.estado_solicitud == TERMINADA: # 3: packages are ready
job.id_paquetes = r.ids_paquetes # only populated here
job.state = JobState.FINISHED
return store.upsert(job)
if r.estado_solicitud in (ERROR, RECHAZADA): # 4, 5: unrecoverable
job.state = JobState.FAILED
return store.upsert(job)
if r.estado_solicitud == VENCIDA: # 6: result expired
job.state = JobState.EXPIRED
return store.upsert(job)
job.state = JobState.IN_PROGRESS # 1 or 2: wait and re-poll
store.upsert(job)
time.sleep(min(base * (2 ** attempt), cap)) # 30s, 60s, 120s, 240s, 300s...
return job # out of attempts; stays IN_PROGRESS, next scheduler tick resumes itDo not hand-roll the raw code mapping if you can avoid it — python-satcfdi or cfdiclient expose helpers, and phpcfdi’s getStatus() gives isAccepted(), isInProgress(), isFinished(), isFailure(), isRejected(), isExpired() plus getCodeRequest(). Polling is the correct model here because the SAT gives you no webhook — the service is asynchronous by design, which is also why requests sit at EnProceso.
Metadata first, CFDI second — and the 72h / 5008 download-once rule
For scoping, request Metadata before full CFDI. Metadata is far lighter (up to 1,000,000 records per request versus 200,000 for CFDI), returns faster, and tells you exactly which UUIDs exist in a window — so you only pull the XML you actually need. A wide-open CFDI request over a busy RFC is what leaves you stuck in EnProceso and risks 5003 (Tope máximo).
Once a request is Terminada, each package is downloadable a maximum of two times — attempt more and you get 5008 (Máximo de descargas permitidas). Combined with the ~72-hour availability window, the rule is simple: download each package once, store it, and never fetch it again. If a package expires before you grab it (6 Vencida), re-queue the window as a brand-new job rather than retrying the dead IdSolicitud.
Incremental sync: a per-RFC watermark, dedup by UUID
The permanent (not just backfill) pipeline advances a per-RFC date watermark and requests only windows after it. Re-requesting a range you already pulled is precisely what manufactures 5005 and then 5002, so the watermark is your guardrail against your own retries.
from datetime import date, timedelta
def next_windows(watermark: date, today: date, *, window_days=7):
"""Yield [start, end] windows strictly AFTER the per-RFC watermark.
We never re-request a range at or before the watermark."""
start = watermark + timedelta(days=1)
while start <= today:
end = min(start + timedelta(days=window_days - 1), today)
yield start, end
start = end + timedelta(days=1)Advance the watermark only after a window is fully downloaded and its CFDI are stored. Storage itself is keyed by the CFDI UUID (folio fiscal), so re-ingesting the same XML — which happens whenever windows overlap at the edges — is a no-op.
INSERT INTO cfdi (uuid, rfc, tipo, role, xml_path, ingested_at)
VALUES (:uuid, :rfc, :tipo, :role, :path, now())
ON CONFLICT (uuid) DO NOTHING;When you need one specific missing invoice rather than a whole window, SolicitaDescargaFolio lets you request by a single UUID instead of a date range — useful for gap-filling without disturbing your watermarks. Everything you store still flows into downstream checks like validating received CFDI.
Scheduling across many RFCs: spread the load, split big ranges, respect the lifetime cap
With thousands of jobs, a scheduler picks work by state, not by client. Run a bounded pool of workers that (1) submit_once any PENDING job, (2) poll_solicitud any REQUESTED/IN_PROGRESS job whose next-poll time has arrived, and (3) download any FINISHED job’s packages. Because every transition is persisted, the pool is safely restartable and horizontally scalable.
Three rules keep you inside the SAT’s limits. Split big ranges into per-week (or per-day) windows so no single request risks 5003 or long EnProceso. Never re-issue a burned window — respect 5002 by treating a lifetime-exhausted query as permanently failed and shifting its boundaries instead of retrying verbatim. And stagger SolicitaDescarga across RFCs rather than firing all clients at once, so you are polling a steady queue instead of thundering the service. Because you submit_once and poll on saved IdSolicituds, adding clients grows the job table, not your duplicate-request risk.
Engineer, not contador — verify, and keep reading
Every code and number above comes from the current SAT web-service spec, but I am an engineer, not your contador or abogado. Confirm the fiscal specifics and any code changes against sat.gob.mx, the reference implementation at phpcfdi/sat-ws-descarga-masiva, and the walkthrough at developers.sw.com.mx.
Keep reading:
- SAT Descarga Masiva — the full status and error codes reference — every
CodEstatusandEstadoSolicituddecoded. - The 4-step Descarga Masiva flow with e.firma, in Python — the single-RFC happy path this pipeline scales up.
- Why your request sits at EnProceso, and the real limits — the async model and the 2,000-CFDI/day myth.
- The CFDI timbrado outbox and idempotency pattern — the durable-job mindset applied to issuing invoices.
- Validating received CFDI (XML, sello, vigencia, 69-B) — what to run on everything you download.
- The Mexican CFDI invoicing hub — the full map of SAT integration guides.