Crawlee v3.1.0 Release Notes
Release Date: 2022-10-13 // over 1 year ago-
๐ Bug Fixes
- โ add overload for
KeyValueStore.getValue
with defaultValue (#1541) (e3cb509) - โ add retry attempts to methods in CLI (#1588) (9142e59)
- ๐ allow
label
inenqueueLinksByClickingElements
options (#1525) (18b7c25) - basic-crawler: handle
request.noRetry
aftererrorHandler
(#1542) (2a2040e) - ๐ build storage classes by using
this
instead of the class (#1596) (2b14eb7) - correct some typing exports (#1527) (4a136e5)
- do not hide stack trace of (retried) Type/Syntax/ReferenceErrors (469b4b5)
- enqueueLinks: ensure the enqueue strategy is respected alongside user patterns (#1509) (2b0eeed)
- enqueueLinks: prevent useless request creations when filtering by user patterns (#1510) (cb8fe36)
- ๐ฆ export
Cookie
fromcrawlee
metapackage (7b02ceb) - ๐ handle redirect cookies (#1521) (2f7fc7c)
- http-crawler: do not hang on POST without payload (#1546) (8c87390)
- โ remove undeclared dependency on core package from puppeteer utils (827ae60)
- ๐ support TypeScript 4.8 (#1507) (4c3a504)
- wait for persist state listeners to run when event manager closes (#1481) (aa550ed)
๐ Features
- โ add
Dataset.exportToValue
(#1553) (acc6344) - โ add
Dataset.getData()
shortcut (522ed6e) - โ add
utils.downloadListOfUrls
to crawlee metapackage (7b33b0a) - โ add
utils.parseOpenGraph()
(#1555) (059f85e) - โ add
utils.playwright.compileScript
(#1559) (2e14162) - โ add
utils.playwright.infiniteScroll
(#1543) (60c8289), closes #1528 - โ add
utils.playwright.saveSnapshot
(#1544) (a4ceef0) - โ add global
useState
helper (#1551) (2b03177) - โ add static
Dataset.exportToValue
(#1564) (a7c17d4) - ๐ allow disabling storage persistence (#1539) (f65e3c6)
- โฌ๏ธ bump puppeteer support to 17.x (#1519) (b97a852)
- core: add
forefront
option toenqueueLinks
helper (f8755b6), closes #1595 - don't close page before calling errorHandler (#1548) (1c8cd82)
- enqueue links by clicking for Playwright (#1545) (3d25ade)
- error tracker (#1467) (6bfe1ce)
- ๐ make the CLI download directly from GitHub (#1540) (3ff398a)
- router: add userdata generic to addHandler (#1547) (19cdf13)
- ๐ use JSON5 for
INPUT.json
to support comments (#1538) (09133ff)
- โ add overload for