Crawlee v3.1.0 Release Notes

Release Date: 2022-10-13 // over 1 year ago
  • ๐Ÿ› Bug Fixes

    • โž• add overload for KeyValueStore.getValue with defaultValue (#1541) (e3cb509)
    • โž• add retry attempts to methods in CLI (#1588) (9142e59)
    • ๐Ÿ‘ allow label in enqueueLinksByClickingElements options (#1525) (18b7c25)
    • basic-crawler: handle request.noRetry after errorHandler (#1542) (2a2040e)
    • ๐Ÿ— build storage classes by using this instead of the class (#1596) (2b14eb7)
    • correct some typing exports (#1527) (4a136e5)
    • do not hide stack trace of (retried) Type/Syntax/ReferenceErrors (469b4b5)
    • enqueueLinks: ensure the enqueue strategy is respected alongside user patterns (#1509) (2b0eeed)
    • enqueueLinks: prevent useless request creations when filtering by user patterns (#1510) (cb8fe36)
    • ๐Ÿ“ฆ export Cookie from crawlee metapackage (7b02ceb)
    • ๐Ÿ– handle redirect cookies (#1521) (2f7fc7c)
    • http-crawler: do not hang on POST without payload (#1546) (8c87390)
    • โœ‚ remove undeclared dependency on core package from puppeteer utils (827ae60)
    • ๐Ÿ‘Œ support TypeScript 4.8 (#1507) (4c3a504)
    • wait for persist state listeners to run when event manager closes (#1481) (aa550ed)

    ๐Ÿ”‹ Features

    • โž• add Dataset.exportToValue (#1553) (acc6344)
    • โž• add Dataset.getData() shortcut (522ed6e)
    • โž• add utils.downloadListOfUrls to crawlee metapackage (7b33b0a)
    • โž• add utils.parseOpenGraph() (#1555) (059f85e)
    • โž• add utils.playwright.compileScript (#1559) (2e14162)
    • โž• add utils.playwright.infiniteScroll (#1543) (60c8289), closes #1528
    • โž• add utils.playwright.saveSnapshot (#1544) (a4ceef0)
    • โž• add global useState helper (#1551) (2b03177)
    • โž• add static Dataset.exportToValue (#1564) (a7c17d4)
    • ๐Ÿ‘ allow disabling storage persistence (#1539) (f65e3c6)
    • โฌ†๏ธ bump puppeteer support to 17.x (#1519) (b97a852)
    • core: add forefront option to enqueueLinks helper (f8755b6), closes #1595
    • don't close page before calling errorHandler (#1548) (1c8cd82)
    • enqueue links by clicking for Playwright (#1545) (3d25ade)
    • error tracker (#1467) (6bfe1ce)
    • ๐Ÿ‘‰ make the CLI download directly from GitHub (#1540) (3ff398a)
    • router: add userdata generic to addHandler (#1547) (19cdf13)
    • ๐Ÿ‘‰ use JSON5 for INPUT.json to support comments (#1538) (09133ff)