Matt Joyce
|
96630154d3
|
Merge pull request #1 from mendableai/main
Fix FIRECRAWL_API_URL bug, also various PyLint fixes
|
2024-05-23 09:16:03 +10:00 |
|
Matt Joyce
|
106c18d11f
|
Use truthiness check for 'success' key in API response
PyLint C0121
|
2024-05-23 08:57:53 +10:00 |
|
Matt Joyce
|
5c21aed9c7
|
adding pylintrc to allow longer lines
|
2024-05-23 08:45:56 +10:00 |
|
Matt Joyce
|
48e91c89e7
|
Removed unnecessary If block
PyLint R1731
|
2024-05-23 08:42:07 +10:00 |
|
Matt Joyce
|
7d2efe5acb
|
Added request timeouts
connection timeout to 5 seconds and the response timeout to 10
PyLint W3101
|
2024-05-23 08:39:19 +10:00 |
|
Matt Joyce
|
96b19172a1
|
Removed trailing whitespace
PyLint C0303: Trailing whitespace (trailing-whitespace)
|
2024-05-23 08:30:23 +10:00 |
|
Matt Joyce
|
6216c85322
|
Time module already imported
Pylint
W0404: Reimport 'time' (imported line 16) (reimported)
C0415: Import outside toplevel (time) (import-outside-toplevel)
|
2024-05-23 08:21:32 +10:00 |
|
Matt Joyce
|
8adf2b7132
|
Added Docstrings for functions
PyLint C0116: Missing function or method docstring (missing-function-docstring)
|
2024-05-23 08:20:32 +10:00 |
|
Matt Joyce
|
971e1f85c4
|
Added module docstring
PyLint C0114 - missing-module-docstring
|
2024-05-23 08:03:58 +10:00 |
|
Matt Joyce
|
8d041c05b4
|
rearranged logic for FIRECRAWL_API_URL
It would not use the ENV unless the param was set to None which was counter-intuitive.
|
2024-05-23 08:00:56 +10:00 |
|
rafaelsideguide
|
aa6df4305e
|
crawl load tests 6 and 7
|
2024-05-22 18:20:24 -03:00 |
|
Nicolas
|
4e39701644
|
Update main.py
|
2024-05-22 12:59:56 -07:00 |
|
rafaelsideguide
|
73f1d09d39
|
Update website_params.ts
|
2024-05-22 15:07:12 -03:00 |
|
Rafael Miller
|
df0550d2f6
|
Merge pull request #143 from mendableai/bug/crawl-limit
[Bug] Fixing /crawl limit
|
2024-05-22 14:51:54 -03:00 |
|
Nicolas
|
3aa5f26627
|
Update main.py
|
2024-05-22 10:45:43 -07:00 |
|
Nicolas
|
3e63985e53
|
Update main.py
|
2024-05-22 10:40:47 -07:00 |
|
rafaelsideguide
|
4dfc371241
|
Update index.test.ts
|
2024-05-22 14:38:41 -03:00 |
|
rafaelsideguide
|
f4a3469b9e
|
Merge branch 'main' into bug/crawl-limit
|
2024-05-22 14:27:28 -03:00 |
|
rafaelsideguide
|
ff147f1f51
|
load testing for crawl
|
2024-05-22 14:26:29 -03:00 |
|
Nicolas
|
0d187f0425
|
Merge pull request #77 from tractorjuice/patch-1
Add additional file extensions to crawler.ts
|
2024-05-22 10:16:49 -07:00 |
|
rafaelsideguide
|
04a0bef0fb
|
Merge branch 'main' into test/load-testing
|
2024-05-22 11:26:19 -03:00 |
|
rafaelsideguide
|
e4573c08ca
|
Update website_params.ts
|
2024-05-22 11:24:48 -03:00 |
|
rafaelsideguide
|
f9ae1729b6
|
Update firecrawl.py
|
2024-05-22 09:40:38 -03:00 |
|
rafaelsideguide
|
068a240ab4
|
load tests for scrape route
|
2024-05-22 09:30:32 -03:00 |
|
Nicolas
|
f915b08016
|
Merge pull request #174 from mendableai/nsc/fire-engine-beta
beta: Fire-Engine fallback
|
2024-05-21 19:25:51 -07:00 |
|
Nicolas
|
cb2bd0e71f
|
Update index.test.ts
|
2024-05-21 19:03:32 -07:00 |
|
Nicolas
|
253abb849f
|
Update rate-limiter.ts
|
2024-05-21 18:53:58 -07:00 |
|
Nicolas
|
229b9908d2
|
Nick: only enable hyper dx in prod
|
2024-05-21 18:52:46 -07:00 |
|
Nicolas
|
a8ff295977
|
Update single_url.ts
|
2024-05-21 18:50:42 -07:00 |
|
Nicolas
|
a5e718b084
|
Nick: improvements
|
2024-05-21 18:34:23 -07:00 |
|
Nicolas
|
2e264a4c75
|
Update ci.yml
|
2024-05-21 13:24:09 -07:00 |
|
Nicolas
|
6285f12cd1
|
Merge pull request #167 from mendableai/nsc/hyper-dx-integration
feat: HyperDX Integration
|
2024-05-21 13:19:38 -07:00 |
|
rafaelsideguide
|
75f4e34d8e
|
Merge branch 'main' into test/load-testing
|
2024-05-21 10:28:02 -03:00 |
|
rafaelsideguide
|
ec46065066
|
Update rate-limiter.ts
|
2024-05-21 10:07:27 -03:00 |
|
rafaelsideguide
|
6a3ac13fe1
|
Update load-test.yml
|
2024-05-21 10:06:02 -03:00 |
|
youqiang
|
c47dae13a9
|
update: wait until body attached in playwright-service
|
2024-05-21 14:53:57 +08:00 |
|
Nicolas
|
7f64fe884a
|
Update blocklist.ts
|
2024-05-20 17:26:01 -07:00 |
|
Nicolas
|
756f54466d
|
Nick: allowed keywords for now
|
2024-05-20 17:24:21 -07:00 |
|
Nicolas
|
01783dc336
|
Update openapi.json
|
2024-05-20 17:10:55 -07:00 |
|
Nicolas
|
77a79b5a79
|
Nick: max num tokens for llm extract (for now) + slice the max
|
2024-05-20 17:07:38 -07:00 |
|
Nicolas
|
2644e1c029
|
Update .env.example
|
2024-05-20 13:36:51 -07:00 |
|
Nicolas
|
9e61d431f0
|
Nick: hyper dx integration init
|
2024-05-20 13:36:34 -07:00 |
|
Nicolas
|
d5d0d48848
|
Merge branch 'main' of https://github.com/mendableai/firecrawl
|
2024-05-20 10:06:52 -07:00 |
|
Nicolas
|
60002e79b8
|
Nick: python sdk bump
|
2024-05-20 10:06:48 -07:00 |
|
Nicolas
|
4bb536e5b3
|
Merge pull request #166 from mattjoyce/main
Allow override of API URL
|
2024-05-20 09:44:47 -07:00 |
|
Matt Joyce
|
7e5ef4dec4
|
Allow override of API URL
Allows python sdk to be used with local installs.
|
2024-05-20 18:46:32 +10:00 |
|
Nicolas
|
c74f757b53
|
Update rate-limiter.ts
|
2024-05-19 13:05:36 -07:00 |
|
Nicolas
|
842e197fd7
|
Merge pull request #151 from mendableai/feat/rate-limits
[Feat] Added rate limits
|
2024-05-19 13:01:37 -07:00 |
|
Nicolas
|
98a39b39ab
|
Nick: increased rate limits
|
2024-05-19 12:59:29 -07:00 |
|
Nicolas
|
18fa15df25
|
Update index.test.ts
|
2024-05-19 12:50:06 -07:00 |
|
