Skip to main content
For AI agents: a documentation index is available at https://docs.parallel.ai/llms.txt. The full text of all docs is at https://docs.parallel.ai/llms-full.txt. You may also fetch any page as Markdown by appending .md to its URL or sending Accept: text/markdown.
The Parallel Task Group API enables you to batch process hundreds or thousands of Tasks efficiently. Instead of running Tasks one by one, you can organize them into groups, monitor their progress collectively, and retrieve results in bulk. The API is comprised of the following endpoints: Creation: To run a batch of tasks in a group, you first need to create a task group, after which you can add runs to it, which will be queued and processed.
  • POST /v1/tasks/groups (Create task-group)
  • POST /v1/tasks/groups/{taskgroup_id}/runs (Add runs. Up to 1,000 runs per POST request.)
Progress Snapshot: At any moment during the task, you can get an instant snapshot of the state of it using GET /{taskgroup_id} and GET /{taskgroup_id}/runs. The runs endpoint streams back the requested runs immediately using SSE to support large payloads without pagination; it does not wait for runs to complete. Task Group runs are subject to your account’s data-retention configuration. Persist any results your application needs after processing; Zero Data Retention (ZDR) deployments do not retain completed run data.
  • GET /v1/tasks/groups/{taskgroup_id} (Get task-group summary)
  • GET /v1/tasks/groups/{taskgroup_id}/runs (Fetch task group runs)
  • GET /v1/tasks/groups/{taskgroup_id}/runs/{run_id} (Retrieve one task run’s status)
Realtime updates: You may want to provide efficient real-time updates to your app. For group status updates and non-active run state events, you can use GET /{taskgroup_id}/events. To retrieve a completed task run’s result, use the task run result endpoint.
  • GET /v1/tasks/groups/{taskgroup_id}/events (Stream task-group events)
  • GET /v1/tasks/runs/{run_id}/result (Get task-run result)
To determine when a task group has no active runs, you can either use realtime update events or poll the task-group summary endpoint. You can continue adding runs to a task group after it becomes inactive.

Key Concepts

Task Groups

A Task Group is a container that organizes multiple task runs. Each group has:
  • A unique taskgroup_id for identification
  • A status object with is_active (boolean) and task_run_status_counts (counts by status)
  • The ability to add new Tasks dynamically

Group Status

Track progress with real-time status updates:
  • Total number of task runs
  • Count of runs by status (queued, action_required, running, completed, failed, cancelling, or cancelled)
  • Whether the group is active (is_active is true while any run is queued, running, or cancelling)
An inactive group has no active runs, but its runs may have completed, failed, been cancelled, or require action. Inspect task_run_status_counts to distinguish those outcomes.

Quick Start

1. Define Types and Task Structure

2. Create a Task Group

3. Add Tasks to the Group

By default, the response refreshes and returns the aggregated status of all runs in the group. If you’re adding tasks at scale, set refresh_status to false to skip the full status refresh — the response still includes a status snapshot from the group’s atomically maintained counters. You can refresh the aggregated status at any time via the GET task-group endpoint.

4. Monitor Progress

5. Retrieve Results

The getRuns endpoint returns a Server-Sent Events stream, not a simple JSON response. It emits one event per run currently in the group (a snapshot of each run’s state), then closes. To pick up runs added after that snapshot, resume from the last event_id via the last_event_id parameter. Each event in the stream has:
  • type: Either "task_run.state" or "error"
  • event_id: Cursor for resuming the stream via the last_event_id parameter
  • run: The TaskRun object with run_id, status, and is_active
  • input: The original input (only included when include_input=true)
  • output: The result output (only included when include_output=true and the run completed successfully)
If you want a live stream of non-active run state transitions and group-level status updates instead of a snapshot, use the /events endpoint shown below the getRuns examples.

Batch Processing Pattern

For large datasets, process Tasks in batches to optimize performance. Setting refresh_status to false is recommended when adding tasks in bulk, as it skips refreshing the group status on each request for faster responses:

Error Handling

The /runs stream is a snapshot and can include active runs. Classify each event using event.run.status, not merely by whether an output is present:

Complete Example

Here’s a complete script that demonstrates the full workflow, including all of the setup code above.