All Versions
35
Latest Version
Avg Release Cycle
68 days
Latest Release
11 days ago

Changelog History
Page 1

  • v3.17.0 Changes

    June 04, 2026

    3.17.0 (2026-06-04)

    🐛 Bug Fixes

    • ➕ add network timeouts to discoverValidSitemaps to prevent indefinite hangs (#3429) (ee90835)
    • 🔌 bind Browser.pages() correctly in PuppeteerPlugin (#3439) (c3a4b3b)
    • 💻 browser-pool: allow unsetting launch context proxy URL (#3621) (d5ab219)
    • 💻 browser-pool: preserve caller's AbortContext across p-limit queue (#3673) (c4d58d9), closes #3670
    • 💻 BrowserCrawler closes ctx.page before errorHandler runs on navig… (#3655) (ab34886)
    • 🔄 change sequential approach to parallel for Iterator first page (#3402) (adf3dae), closes #3395
    • core: apply object searchParams in HTTP request options (#3622) (d32946e)
    • core: release storage open queue slot on failure (#3662) (18ebc61), closes #3661
    • 🌲 correct failed requests delta count in periodic log message (#3482) (a098f42)
    • Correctly track the number of requests handled by a crawler (#3410) (b2296ce), closes #3083 #3330
    • e2e: resolve flaky Camoufox tests due to GitHub rate limits (#3527) (08c98cf), closes #3524
    • honor custom respectRobotsTxtFile userAgent in enqueueLinks (#3578) (1d2528b)
    • ⚠ log warning when Sitemap.load returns empty sitemap due to loading error (#3462) (2cf9c1c)
    • 📜 parse proxy URLs correctly when using useIncognitoPages (#3433) (db2bb68)
    • Prevent accidental request dropping with maxRequestsPerCrawl (#3531) (b23319b)
    • prevent iframe expansion failure on pages with Trusted Types CSP (#3590) (c0b9b50), closes #3588
    • prevent v3 sitemap discovery init hangs in discoverValidSitemaps (#3434) (6c04f92), closes #3412
    • 💻 unref BrowserController.close() fallback timer to avoid keeping Node alive longer than needed (#3671) (eef94d4)
    • using GET for candidate checks - sitemaps (#3464) (f82eb04), closes #3463

    🔋 Features

    • adaptive-crawler: add shouldPropagateError callback to prevent browser fallback (#3635) (d3a29d9)
    • ➕ add ignoreProxyCertificate option for the internal proxy-chain instance (#3418) (02eec66), closes #3369
    • ➕ add abortDownload context helper to FileDownload (#3528) (3b3d8f6)
    • Dynamic memory snapshots (#3471) (87c67dc), closes #3408
    • 🔦 expose Download objects on PlaywrightCrawlingContext (#3596) (5dc1627), closes #3583
    • Modularize SystemStatus to allow custom backpressure mechanisms (#3529) (7159a13)
    • templates: use ImpitHttpClient and install optional deps in Docker (#3603) (113dca6), closes #3601
    • utils: add sitemapFilter option to parseSitemap (#3557) (1d4f6b9)

    🐎 Performance Improvements

    • ➕ add lazyIterablePromise to kvs iterators (#3413) (2658610)

  • v3.16.0 Changes

    February 06, 2026

    3.16.0 (2026-02-06)

    🐛 Bug Fixes

    • adaptive-crawler: Fix persistence of rendering type detection results (#3368) (4abca8b)
    • certain redirect responses change request methods (#3296) (efac644), closes #2586
    • clean turbo cache and tsbuildinfo files in yarn clean (#3348) (6cd9456)
    • core: ensure maxCrawlDepth warning is logged only once (#3337) (9d01334), closes #3336
    • 🖐 handle multiple BasicCrawler.stop() calls correctly (#3324) (9c0580b), closes #3257
    • impit-client: pause fromWeb stream to prevent early consumption (#3347) (72aacb4), closes #555
    • more permissive accept (#3373) (d03af1b), closes #3242
    • ✂ remove deprecation from RequestQueueV1 (#3341) (89309bc)
    • suppress info message for undefined maxRequestsPerCrawl (#3237) (f3d9a79)

    🔋 Features

    • ➕ add @crawlee/stagehand package for AI-powered browser automation (#3331) (a89cb5a), closes #3064
    • ➕ Add a counter of in-flight rendering type detections (#3355) (565fc34)
    • implements async iterators (#3352) (7f7a4ab), closes #3338
    • 🔧 make handleCloudflareChallenge more configurable (#3247) (629daf8), closes #3127
    • utils: add discoverValidSitemaps utility (#3339) (29f52ed)

    🐎 Performance Improvements

  • v3.15.3 Changes

    November 10, 2025

    3.15.3 (2025-11-10)

    🐛 Bug Fixes

    🔋 Features

    • 👌 support custom userAgent with respectRobotsTxtFile (#3226) (354252d), closes #3222

  • v3.15.2 Changes

    October 23, 2025

    3.15.2 (2025-10-23)

    🐛 Bug Fixes

    🔋 Features

    • 📦 export cheerio types in all crawler packages (#3204) (f05790b)

    🐎 Performance Improvements

  • v3.15.1 Changes

    September 26, 2025

    3.15.1 (2025-09-26)

    🐛 Bug Fixes

    • 👉 use correct config for storage classes to avoid memory leaks (#3144) (911a2eb)

    🐎 Performance Improvements

    • 👌 Improve glob performance by reusing minimatch objects (#3168) (e5632e2)

  • v3.15.0 Changes

    September 17, 2025

    3.15.0 (2025-09-17)

    🐛 Bug Fixes

    • ⏱ ImpitHttpClient respects the internal Request timeout (#3103) (a35376d)
    • proxyUrls list can contain null (#3142) (dc39cc2), closes #3136
    • don't fail exportData calls on empty datasets (#3115) (298f170), closes #2734
    • respect maxCrawlDepth with a custom enqueueLinks transformRequestFunction (#3159) (e2ecb74)

    🔋 Features

    • ➕ add collectAllKeys option for BasicCrawler.exportData (#3129) (2ddfc9c), closes #3007
    • ➕ add TandemRequestProvider for combined RequestList and RequestQueue usage (#2914) (4ca450f), closes #2499

  • v3.14.1 Changes

    August 05, 2025

    3.14.1 (2025-08-05)

    Note: Version bump only for package @crawlee/root

  • v3.14.0 Changes

    July 25, 2025

    3.14.0 (2025-07-25)

    🐛 Bug Fixes

    • 💻 don't retire browsers with long-running pre|postLaunchHooks prematurely (#3062) (681660e)
    • respect exclude option in enqueueLinksByClickingElements (#3058) (013eb02)
    • retry on blocked status codes in HttpCrawler (#3060) (b5fcd79), closes #3029
    • validation of iterables when adding requests to the queue (#3091) (529a1dd), closes #3063

    🔋 Features

  • v3.13.10 Changes

    July 09, 2025

    3.13.10 (2025-07-09)

    🐛 Bug Fixes

  • v3.13.9 Changes

    June 27, 2025

    3.13.9 (2025-06-27)

    🐛 Bug Fixes

    • 🌲 Do not log 'malformed sitemap content' on network errors in Sitemap.tryCommonNames (#3015) (64a090f), closes #2884
    • 🛠 Fix link filtering in enqueueLinks in AdaptivePlaywrightCrawler (#3021) (8a3b6f8), closes #2525

    🔋 Features

    • Accept (Async)Iterables in addRequests methods (#3013) (a4ab748), closes #2980
    • Report links skipped because of various filter conditions (#3026) (5a867bc), closes #3016