Fetch only the records you need
SEC’s submissions endpoint uses a zero-padded ten-digit CIK. Its recent filing arrays include form, filing date, accession, primary document and registration file number. Historical files referenced by the response may be needed for older research; the pilot’s chosen histories are present in the captured recent arrays.
The tested maintenance pipeline saves the original JSON, archives selected original prospectus HTML, and records retrieval time and content hashes. Coverage still depends on the manually selected cohort.
Minimal selected-registration example
# Illustrative selected-company fetch; use your own monitored contact address.
import json, time, urllib.request
cik = 2130357
request = urllib.request.Request(
f"https://data.sec.gov/submissions/CIK{cik:010}.json",
headers={"User-Agent": "Your Company research@example.com"},
)
with urllib.request.urlopen(request, timeout=30) as response:
company = json.load(response)
recent = company["filings"]["recent"]
for i, form in enumerate(recent["form"]):
if recent["fileNumber"][i] == "333-299633":
print(recent["filingDate"][i], form, recent["accessionNumber"][i])
time.sleep(0.6) # Also coordinate request rates across processes.
Illustrative excerpt of the retrieval pattern. Replace the example contact address. The full pilot pipeline additionally archives sources and builds the tables.
Operational checks
- Declare an identifiable User-Agent and obey SEC fair-access rules. SEC currently states a maximum of 10 requests per second; this pilot uses serial requests separated by at least 0.55 seconds.
- Stop on access-control or rate-limit errors; do not rotate identities to bypass them.
- Do not expose the refresh call directly to arbitrary public requests. Cache a scheduled or manually reviewed result.
- Keep a manual review date separate from automatic retrieval time.
- Confirm from the document cover that a selected 424B4 belongs to the IPO rather than a later offering.
Source notes
- SEC submissions API ↗
Public filing metadata and historical file structure
- SEC fair access ↗
Declared User-Agent and request limits
The Synoptic workflow
Explore the Synoptic workflow
See how this topic connects to source-linked streams, APIs and research tools.
Explore the Synoptic workflow