Why does the API return only 10 or 100 items, or time out when I fetch everything?

Short answer: List endpoints return one page at a time. Loop through pages with page_number and a sensible page_size instead of requesting everything at once.

Why it happens

• List endpoints are paginated. List all dataset records returns 10 records per page by default.
• With apply_preference=true, List all process items (Admin) applies the process's saved preferences, including filters, shown fields and rows per page. That can cap the page at 100.
• A huge page size, such as page_size=9999999, makes the server try to return everything at once. The request drops with a 503 or times out with a 524.
• Filters saved on a list in the UI, such as portal users, also apply to the API response.

How to fix it

1. Request one page at a time, for example ?page_number=1&page_size=1000, then page_number=2, and so on, until a page comes back short or empty.
2. For very large sets, add a short delay between calls.
3. Leave out apply_preference unless you want the saved filters and page size.
4. If a list returns fewer records than you expect, check for filters saved on that list or dataset in the UI.

If you call the API from a page or form with kf.api():

1. Remember that kf.api() calls time out after 10 seconds, so keep pages small.
2. List all process items (Admin) needs the Process Admin role. A non-admin user who clicks a button that calls it gets a permission error. Run the call from an integration instead, for example one started by a webhook.

See API and SDK documentation for each endpoint's parameters.