Skip to content

docs: align expert course with the updated Amazon starter repo - #3137

Draft
honzajavorek wants to merge 3 commits into
masterfrom
claude/gallant-bell-8df5pm
Draft

honzajavorek wants to merge 3 commits into
masterfrom
claude/gallant-bell-8df5pm

Conversation

@honzajavorek

@honzajavorek honzajavorek commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

The expert course now builds on the fixed academy-amazon-scraper starter repo instead of the archived course. Solution code follows the starter's layout and code style, including the Actor template alignment: Dockerfile in the root, @crawlee/cheerio imports, and context pushData/addRequests. demo-actor is renamed to match the starter, and the filter Actor parses prices like $1,080.59 correctly. Closes #3122.

Still to do before this is ready:

  • Run each solution on top of the starter repo and confirm it works (code was only checked against the repo's source, not executed)
  • Check that screenshots still match Amazon and Apify Console: images/product-offers.jpg, solutions/images/adding-webhook.jpg, webhook-succeeded.png, dataset-preview.png, result-items.jpg, create-new-task.jpg, creating-task.png
  • Decide whether the tasks lesson should explicitly ask readers to create the academy-amazon-scraper-task task, which the API & client solution relies on
  • Revisit the solution code once the starter moves to Crawlee 4 and Apify SDK 4 (Port to Crawlee 4 and Apify SDK 4 apify-projects/academy-amazon-scraper#3)

🤖 Generated with Claude Code

https://claude.ai/code/session_01M4cTCT4ARqx59nHGuyeXAw

The academy-amazon-scraper starter repo now scrapes all offers per
product, retries empty pages, and pins proxies to the US. Update the
expert course to build on it instead of the archived course, and bring
the solution code in line with its routes and ES modules.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M4cTCT4ARqx59nHGuyeXAw
@apify-service-account

apify-service-account commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

✅ Preview for this PR (commit d07bf39e) is ready at https://pr-3137.preview.docs.apify.com (see action run).

The starter repo now matches the js-crawlee-cheerio template: the
Dockerfile lives in the project root, code imports from @crawlee/cheerio,
and handlers use pushData and addRequests from the crawling context.
Update the course to match, and fix the input schema paths and prefill
fields of the Actors readers create.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M4cTCT4ARqx59nHGuyeXAw
@honzajavorek

Copy link
Copy Markdown
Collaborator Author

I ran the solutions locally on top of the starter (apify run, keyword headphones):

  • V Handling migrations: works. Killed a run midway and resumed it; the tracker counts match the dataset (71/71).
  • VI Rotating proxies: works. The saved session pool shows maxUsageCount: 5 and maxErrorScore: 1.
  • VII Saving run stats: works. totalSaved matches the item count, errors are recorded, and dateHandled, numberOfRetries, and currentPendingRequests are filled in.
  • I Integrating webhooks: the filter Actor works on a real dataset (cheapest offer per ASIN, $1,080.59 parses correctly). I didn't test the webhook itself on the platform.
  • IV API & client: not run, because it needs the task on the platform.

Issues:

  1. VII: errorHandler: async ({ error, request }) only works because Crawlee 3 still adds a deprecated error to the context. The documented signature is async ({ request }, error) => ….
  2. VII adds persistStateKey: 'AMAZON-SESSIONS' without explaining it, and solution VI doesn't have it.
  3. V imports ASIN_TRACKER from constants.js but never shows its definition (export const ASIN_TRACKER = 'ASIN-TRACKER';).
  4. The new asinTracker.js and Stats.js fail the starter's npm run lint: console.log, default exports, and import order.
  5. Starter, not this PR: the default keyword iphone returns only renewed iPhones with "No featured offers available", so a run with the default input produces an empty dataset.

Amazon blocks the search page after many runs in a row. A few runs failed on all retries, and a short pause fixed it.

Copy link
Copy Markdown
Collaborator Author

Follow-up: I tested I and IV on the platform. The starter was deployed as is, filter-actor and actor-caller were built from the code and schemas in the docs, and the keyword was headphones.

  • I Integrating webhooks: works after one fix. The webhook fired after the scraper succeeded and started filter-actor with the scraper's dataset ID. The filter run returned 16 items for 16 ASINs, each with that ASIN's lowest price (120 scraped offers, 0 mismatches).
  • IV API & client: works after one fix. Both modes write OUTPUT as text/csv with only the requested fields and exactly maxItems rows (10 with the client, 5 with the API).

Issues:

  1. I and IV: new Actors run with limited permissions by default, and the docs don't mention it.
    • filter-actor fails with 403 insufficient-permissions when it reads the scraper's dataset. Declaring the input as a dataset resource fixes it without full permissions. Both properties are needed; resourceType alone still gets a 403:
      "datasetId": { "title": "Dataset ID", "type": "string", "description": "…", "editor": "textfield", "resourceType": "dataset", "resourcePermissions": ["READ"] }
    • actor-caller fails with 403 insufficient-permissions when it starts the task. It worked after switching it to Settings > Permissions > Full permissions. The lesson should tell readers to do that.
  2. IV: task.call({ memory }) doesn't set memory. The first argument of call() is an input override, so the task got {"keyword":"headphones","memory":4096} as input and ran with the default 1024 MB. Use task.call(undefined, { memory }). The API version passes memory correctly, and that task run got 4096 MB.
  3. IV: nothing in the course tells readers to create a task named academy-amazon-scraper-task, and the solution only works if one exists. I created it by hand with input {"keyword": "headphones"}.
  4. IV: the prose says the result is saved under OUTPUT.csv, but the code saves it under OUTPUT.
  5. I: the payload template appears only in the screenshot. It needs to be {"datasetId": {{resource.defaultDatasetId}}} with no quotes around the variable. With quotes, the filter Actor receives the literal string {{resource.defaultDatasetId}} and fails. Writing the template as text would make it copyable and harder to get wrong.
  6. Minor: the client's CSV starts with a byte order mark (BOM), and the API's CSV doesn't.

Amazon didn't block any of the five platform runs.


Generated by Claude Code

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Expert course: the academy-amazon-scraper starter repo is outdated

3 participants