0
Commit Graph

253 Commits

Author SHA1 Message Date
Nicolas
bfccaf670d Nick: fixes most of it 2024-05-15 15:30:37 -07:00
rafaelsideguide
d91043376c not working yet 2024-05-15 18:54:40 -03:00
rafaelsideguide
fa014defc7 Fixing child links only bug 2024-05-15 18:35:09 -03:00
rafaelsideguide
da8d94105d fixed for testing the crawl algorithm only 2024-05-15 17:16:03 -03:00
Nicolas
95ffaa2236 Update crawl.test.ts 2024-05-15 12:58:02 -07:00
Nicolas
f15b8f855e Update crawl.json 2024-05-15 12:57:24 -07:00
Nicolas
98dd672d0a Update crawl.json 2024-05-15 12:55:04 -07:00
Nicolas
499671c87f Update crawl.test.ts 2024-05-15 12:50:13 -07:00
Nicolas
58053eb423 Update rate-limiter.ts 2024-05-15 12:47:35 -07:00
Nicolas
4745d114be Update crawl.test.ts 2024-05-15 12:42:14 -07:00
Nicolas
1601e93d69 Merge branch 'main' into test/crawl-options 2024-05-15 12:34:47 -07:00
Nicolas
3678d3c986 Merge branch 'main' of https://github.com/mendableai/firecrawl 2024-05-15 12:11:18 -07:00
Nicolas
fd82982a31 Nick: 2024-05-15 12:11:16 -07:00
rafaelsideguide
4925ee59f6 added crawl test suite 2024-05-15 15:50:50 -03:00
Nicolas
1b0d6341d3 Update index.ts 2024-05-15 11:48:12 -07:00
Nicolas
d10f81e7fe Nick: fixes 2024-05-15 11:28:20 -07:00
Nicolas
87570bdfa1 Update index.ts 2024-05-15 11:06:03 -07:00
Nicolas
27e1e22a0a Update index.test.ts 2024-05-14 12:28:25 -07:00
Nicolas
a0fdc6f7c6 Nick: 2024-05-14 12:12:40 -07:00
Nicolas
7f31959be7 Nick: 2024-05-14 12:04:36 -07:00
Nicolas
8a72cf556b Nick: 2024-05-13 21:10:58 -07:00
Nicolas
a96fc5b96d Nick: 4x speed 2024-05-13 20:45:11 -07:00
Nicolas
e26008a833 Merge branch 'main' of https://github.com/mendableai/firecrawl 2024-05-13 19:54:13 -07:00
Nicolas
512449e1aa Nick: v21 2024-05-13 19:54:12 -07:00
Nicolas
bd27b0e17e
Merge pull request #142 from mendableai/doc/crawl-limit-default
[Doc] Added default value for crawlOptions.limit
2024-05-13 18:38:09 -07:00
Nicolas
aa0c8188c9 Nick: 408 handling 2024-05-13 18:34:00 -07:00
Nicolas
999176d576 Merge branch 'main' of https://github.com/mendableai/firecrawl 2024-05-13 13:57:34 -07:00
Nicolas
f3ec21d9c4 Update runWebScraper.ts 2024-05-13 13:57:22 -07:00
Nicolas
65d89afba9 Nick: 2024-05-13 13:01:43 -07:00
Eric Ciarla
4cc46d4af8 Update models.ts 2024-05-13 15:23:31 -04:00
Nicolas
2ce045912f Nick: disable vision right now 2024-05-13 10:56:08 -07:00
rafaelsideguide
df16890f84 Added default value for crawlOptions.limit 2024-05-10 11:59:33 -03:00
Nicolas
66bd1e4020 Update website_params.ts 2024-05-09 18:41:15 -07:00
Nicolas
c02a82c282 Update main.py 2024-05-09 18:02:34 -07:00
Nicolas
efc6fcb474 Merge branch 'main' of https://github.com/mendableai/firecrawl 2024-05-09 18:01:04 -07:00
Nicolas
73687822ad Update main.py 2024-05-09 18:00:58 -07:00
Nicolas
d21091bb06 Update single_url.ts 2024-05-09 17:52:46 -07:00
Nicolas
be85008622 Nick: better 2024-05-09 17:48:11 -07:00
Nicolas
be5661a768 Nick: a lot better 2024-05-09 17:45:16 -07:00
Nicolas
fce17e6beb Update credit_billing.ts 2024-05-09 15:29:58 -07:00
rafaelsideguide
f4d8b2c89a Updated docs 2024-05-09 10:36:56 -03:00
Nicolas
aa6b84c5fa Nick: readme 2024-05-08 17:41:15 -07:00
Nicolas
d9da4b53f8 Update example.py 2024-05-08 17:36:40 -07:00
Nicolas
4c88d5da66 Nick: v8 python 2024-05-08 17:35:16 -07:00
Nicolas
e6dbbf1bab Nick: fixes js and pydantic implementation 2024-05-08 17:16:59 -07:00
Nicolas
c89964b230 Nick: 2024-05-08 16:38:49 -07:00
Nicolas
9541ff6b30 Nick: 429 addressed 2024-05-08 15:14:39 -07:00
Nicolas
3bfef646e0 Update index.test.ts 2024-05-08 13:23:53 -07:00
Nicolas
6ced8e73a7 Update index.test.ts 2024-05-08 13:13:38 -07:00
Nicolas
c50076c377 Update websites.json 2024-05-08 13:04:17 -07:00