Page through a large catalog

Uses: Python · TypeScript · CLI · REST API

List endpoints are offset-based. Ask for a page with offset and top, and read totalCount to know when you’re done.

cURL samples assume BASE=https://api.engini.io/v1 and AUTH="x-api-key: $ENGINI_API_KEY" - the setup from the REST walkthrough. Manual paging is a REST-only concern: the SDKs and the CLI page for you, which is why their tabs below are one-liners.

{ "items": [ ... ], "totalCount": 1374, "offset": 0, "top": 100 }

The loop

# the SDK already does this - list methods return the complete set
tools = client.tools.get(applications=["monday"])
apps  = client.applications.list(available=True)

const tools = await client.tools.get({ applications: ["monday"] });
const apps = await client.applications.list({ available: true });

# the CLI pages for you; --limit caps the result, not the page size
engini tools list --application monday --limit 500 | jq -r '.[].tool_slug'
engini applications list --available | jq 'length'

offset=0
while :; do
  page=$(curl -s "$BASE/tools?offset=$offset&top=100" -H "$AUTH")
  echo "$page" | jq -r '.items[].toolSlug'
  count=$(echo "$page" | jq '.items | length')
  total=$(echo "$page" | jq '.totalCount')
  offset=$((offset + count))
  { [ "$count" -eq 0 ] || [ "$offset" -ge "$total" ]; } && break
done

Terminate on both conditions - an empty page and offset >= totalCount. Checking only totalCount can loop forever if a page comes back short; checking only emptiness costs an extra request every time.

Defaults worth knowing

Endpoint	`top` default	Max
`GET /v1/applications`	100	-
`GET /v1/tools`	100	-
`GET /v1/toolsets`	100	100
`GET /v1/connections`	25	100
`GET /v1/connections/{id}/objects`	100	1000

Connections defaulting to 25 catches people out - a naive single request looks like the full list when it isn’t.

Filter before you page

Almost always faster than walking everything:

tools = client.tools.get(applications=["monday"])
apps  = client.applications.list(available=True, search="crm")
conns = client.connections.list(application="gmail")

const tools = await client.tools.get({ applications: ["monday"] });
const apps = await client.applications.list({ available: true, search: "crm" });
const conns = await client.connections.list("gmail");

engini tools list --application monday
engini applications list --available --search crm
engini connections list --application gmail

curl -s "$BASE/tools?applicationSlug=monday&top=100" -H "$AUTH"
curl -s "$BASE/applications?available=true&search=crm" -H "$AUTH"
curl -s "$BASE/connections?applicationSlug=gmail" -H "$AUTH"

?search= runs a semantic search and is billed against a separate, stricter rate-limit budget than ordinary reads. It’s the right tool for “find me something like X”, but don’t put it inside a pagination loop - filter with applicationSlug or name there instead.

Rate limits while paging

Two budgets apply per account - a per-second burst and an hourly window. A tight loop over a large catalog can trip the burst, so handle 429 rather than assuming it won’t happen:

resp=$(curl -s -w '\n%{http_code}' "$BASE/tools?offset=$offset&top=100" -H "$AUTH")
code=$(echo "$resp" | tail -1)
if [ "$code" = "429" ]; then
  sleep "$(curl -sI "$BASE/tools" -H "$AUTH" | awk -F': ' '/[Rr]etry-[Aa]fter/{print $2+0}')"
  continue
fi

Retry-After is in seconds: about 1 when the burst window trips, 3600 when the hourly one does. The difference matters - a one-second pause and an hour-long wait deserve very different handling. In the SDKs the same 429 surfaces as a typed EnginiRateLimitError - neither SDK retries for you, by design. See SDK errors & pagination for the retry pattern.

Start here

Connect apps

Run tools

Build agents

Automate

The loop

Defaults worth knowing

Filter before you page

Rate limits while paging

​The loop

​Defaults worth knowing

​Filter before you page

​Rate limits while paging

The loop

Defaults worth knowing

Filter before you page

Rate limits while paging