Oncrawl Crawl Resolver

The Oncrawl Crawl Resolver node resolves one completed Oncrawl crawl and returns its id and dates, so a workflow always reads fresh data and records which crawl produced it.

What does the Oncrawl Crawl Resolver node do?#

Oncrawl does not publish a continuous stream of data: it publishes crawls. Each crawl is a separate dataset with its own id, and a new one appears every time the crawl schedule runs. That creates two practical problems in a workflow. First, a hardcoded crawl id goes stale the moment the next crawl finishes. Second, a metric read from “Oncrawl” means nothing on its own, because the number came from one specific crawl and nobody recorded which.

The Oncrawl Crawl Resolver solves both. It picks completed crawls — the newest one by default — and returns their identity: crawl_id, project_id, the crawl name, and the created and ended timestamps. Downstream nodes read the crawl_id, so the workflow follows the latest crawl without being edited, and the run history keeps the timestamps, so the result stays interpretable later.

Only crawls with the status done are returned. A running or failed crawl has no queryable dataset, and letting one through would produce an empty result further down the workflow instead of a clear error.

Common use cases:

• Feeding a fresh crawl_id to the Oncrawl Data Reader or the Oncrawl KPI Reader on every scheduled run.
• Comparing crawls in the same run: one resolver set to Latest 2 crawls returns the newest and the previous one.
• Recording the crawl date in a report so a reader knows how old the data is.
• Branching on crawl freshness before launching an expensive analysis.

Quick setup#

Configuration parameters#

Required fields#

Optional fields#

Output#

The node exposes two outputs, because a reader and a report need different things.

One crawl:

{
  "crawl_id": "abc123",
  "project_id": "p1",
  "crawl_name": "Weekly crawl",
  "status": "done",
  "created_at": 1720000000000,
  "ended_at": 1720003600000,
  "retrieved_at": "2026-10-20T14:00:00+00:00",
  "source": "Oncrawl API v2",
  "available_completed_crawls": 12
}

Several crawls, newest first:

{
  "crawl_id": ["abc123", "def456"],
  "project_id": "p1",
  "requested_crawls": 2,
  "resolved_crawls": 2,
  "available_completed_crawls": 12,
  "crawls": [
    { "crawl_id": "abc123", "ended_at": 1720003600000, "status": "done" },
    { "crawl_id": "def456", "ended_at": 1719398800000, "status": "done" }
  ],
  "retrieved_at": "2026-10-20T14:00:00+00:00",
  "source": "Oncrawl API v2"
}

available_completed_crawls is returned in the Latest crawl modes only. A node that reads one crawl takes the first id; the Oncrawl KPI Reader reports on every crawl of the selection.

Troubleshooting#

No completed Oncrawl crawl is available for this project

Cause: The project exists but has no crawl with the status done — typically a project whose first crawl is still running.

Solution: Wait for the crawl to finish, or pick a project that already has a completed crawl.

Fewer crawls were resolved than the mode asked for

Cause: The project has a shorter crawl history than the selected mode — Latest 5 crawls on a project with three completed crawls.

Solution: Nothing is wrong, and the run does not fail: requested_crawls and resolved_crawls report the difference in the payload. Pick a smaller mode if a downstream node needs an exact count.

Oncrawl rejected the token as unauthenticated (401)

Cause: The token is wrong or has been revoked. A valid token that is not allowed the action answers 403, not 401, so a 401 means the token itself was not recognised.

Solution: Create a new token in Oncrawl under Settings > Tokens and reconnect the integration.

Oncrawl refused the request (403)

Cause: Oncrawl recognised the token but refused the action. The message names the reason Oncrawl gave:

• quota_error — the account’s quota does not allow the request.
• no_active_subscription — the account has no active subscription.
• feature_not_available — the plan does not include this feature, for example the API option.
• otherwise — the token lacks the projects:read scope, or access to this project.

Solution: Free up or raise the quota, renew the subscription or the plan, or regenerate the token with the projects:read scope on an account that can see the project.

Oncrawl rate limit exceeded (429)

Cause: Oncrawl answered 429 to three attempts in a row. The node retries a 429 or a 5xx error twice, a few seconds apart, before giving up. Oncrawl does not publish its rate limit.

Solution: Re-run later, and avoid starting several workflows that read Oncrawl at the same moment.

Best practices and pitfalls#

How does it fit into a workflow?#

The resolver sits at the head of an Oncrawl chain, and a multi-crawl mode turns the same chain into a comparison:

graph LR
    R[Oncrawl Crawl Resolver<br/>Latest 2 crawls] --> K[Oncrawl KPI Reader<br/>both crawls]
    K --> LLM[LLM<br/>compare the two crawls]
    LLM --> Email[Email Sender]

Related nodes#