Nicolas
beb7526d1d
Update webhook.ts
2024-06-05 10:38:05 -07:00
Nicolas
1a16378fe8
Merge pull request #234 from JakobStadlhuber/feat/webhook-self-hosted
...
Add support for Self-Hosted Webhook URL Usage and added project_id into the webhook payload
2024-06-05 10:25:05 -07:00
Nicolas
7cb14edec8
Nick:
2024-06-05 10:13:52 -07:00
Rafael Miller
9e000ded03
Merge branch 'main' into feat/better-gdrive-pdf-fetch
2024-06-05 14:07:56 -03:00
rafaelsideguide
ccc55127d6
Added scroll xpaths on fire-engine for handling readme docs
2024-06-05 11:48:41 -03:00
rafaelsideguide
b5045d1661
[feat] improved the scrape for gdrive pdfs
2024-06-04 17:47:28 -03:00
Nicolas
96257b7b17
Update handleCustomScraping.ts
2024-06-04 12:22:46 -07:00
Nicolas
674500affa
Nick:
2024-06-04 12:15:39 -07:00
rafaelsideguide
5ae4d1caf5
Update single_url.ts
2024-06-04 15:28:09 -03:00
Jakob Stadlhuber
9e5ddec207
Remove default webhook URL from .env.example
...
The default value for the SELF_HOSTED_WEBHOOK_URL in the .env.example file was removed to prevent unintentional exposure or usage. The users are now required to explicitly specify
2024-06-04 19:56:35 +02:00
Jakob Stadlhuber
6208f4207d
Add support for Self-Hosted Webhook URL Usage and added project_id into the webhook payload
...
This commit introduces the capability of using a Self-Hosted Webhook URL. The application now checks for a self-hosted URL before querying the database for the webhook settings. If a Self-Hosted Webhook URL is set in the environment variables, it will be used directly, diminishing unnecessary database queries.
2024-06-04 19:55:07 +02:00
rafaelsideguide
93f3098672
build files
2024-06-04 14:54:54 -03:00
rafaelsideguide
64a4338ff0
Update single_url.ts
2024-06-04 14:40:05 -03:00
Rafael Miller
02fe470e20
Merge pull request #148 from mendableai/nsc/improvemnts-fixes-misc
...
Better fallbacks for initial crawl start
2024-06-04 14:31:10 -03:00
Rafael Miller
665a40d9f4
Merge pull request #212 from mendableai/bugfix/partial-data-js-sdk
...
[Bug] Improved js response and test for getting partial_data
2024-06-04 14:05:23 -03:00
rafaelsideguide
1f4c6b7a87
Update package.json
2024-06-04 13:59:48 -03:00
Rafael Miller
19c67916d4
Merge pull request #211 from mendableai/fix/rename-variables
...
[Fix] Changed timeout parameter name on js sdk
2024-06-04 13:57:58 -03:00
Rafael Miller
f4f87b5374
Merge branch 'main' into bugfix/partial-data-js-sdk
2024-06-04 13:40:42 -03:00
rafaelsideguide
4e3a0495d7
updated version 0.0.12 -> 0.0.13
...
- [ ] publish
2024-06-04 12:03:55 -03:00
Rafael Miller
b80fb374e5
Merge branch 'main' into playwright-service-bug-222
2024-06-04 11:57:17 -03:00
rafaelsideguide
6920ec8a61
bugfixing. already on main
2024-06-04 11:05:50 -03:00
Nicolas
d91b725c6f
Update fly.toml
2024-06-04 00:41:15 -07:00
Nicolas
cbf8d79cce
Update pdfProcessor.ts
2024-06-04 00:13:37 -07:00
Nicolas
3fc9004ba8
Update fly.toml
2024-06-03 23:49:46 -07:00
Nicolas
2ea01f1456
Update single_url.ts
2024-06-03 23:42:39 -07:00
Nicolas
854d5b3cb3
Update single_url.ts
2024-06-03 23:32:55 -07:00
Nicolas
99059814a8
Nick:
2024-06-03 21:32:48 -07:00
Nicolas
918059ee9e
Merge branch 'main' into nsc/improvemnts-fixes-misc
2024-06-03 16:46:02 -07:00
Nicolas
38e583f66c
Update socialBlockList.test.ts
2024-06-03 16:44:23 -07:00
Nicolas
c69c89f838
Nick:
2024-06-03 16:42:42 -07:00
Nicolas
48d1ec05b2
Merge branch 'main' into nsc/improved-blocklist
2024-06-03 16:38:03 -07:00
Nicolas
d30ced4394
Merge pull request #221 from mendableai/nsc/fwd-header-auth
...
feat: Ability to forward headers to reliable providers for auth etc...
2024-06-03 16:33:40 -07:00
Romain Bruyère
4987f901d1
Merge branch 'mendableai:main' into main
2024-06-03 21:29:33 +02:00
rafaelsideguide
4100cc9223
Update index.test.ts
2024-06-03 16:29:16 -03:00
rombru
3ff91ddd1f
fix: use @ instead of # for default BULL_AUTH_KEY. hash mark is reserved for URI fragments.
2024-06-03 21:28:25 +02:00
rafaelsideguide
c1aed1360e
Update index.test.ts
2024-06-03 15:51:07 -03:00
rafaelsideguide
1fc3a15149
Update single_url.ts
2024-06-03 15:24:40 -03:00
Nicolas
fde522c3e1
Update single_url.ts
2024-06-02 20:23:45 -07:00
Matt Joyce
deefe65cbe
Change the way the playwright response is parsed
...
Was failing with a Type Error, but actually looked ok.
This fixes the type error, and stop scraper fallback.
2024-06-01 19:16:56 +10:00
Matt Joyce
14896a9fdd
Fix PLAYWRIGHT_MICROSERVICE_URL
...
It needs to end in html, otherwise scrape will 404
2024-06-01 19:03:16 +10:00
Matt Joyce
1eacad4ef3
Clarifying wait type and name
2024-06-01 18:53:03 +10:00
Matt Joyce
c516140bfb
Various Linting
...
Pylint
C0114: Missing module docstring
C0115: Missing class docstring
C0116: Missing function or method docstring
C0303: Trailing whitespace
Import ordering
2024-06-01 18:53:03 +10:00
Matt Joyce
2a39b5382b
Add timeout to class and provide default.
2024-06-01 18:52:42 +10:00
Nicolas
8cb62dde92
Update website_params.ts
2024-05-31 16:09:39 -07:00
Nicolas
3b8059edb6
Update single_url.ts
2024-05-31 15:43:06 -07:00
Nicolas
6bea803120
Nick:
2024-05-31 15:39:54 -07:00
Nicolas
2139129296
Nick: v12
2024-05-31 11:39:55 -07:00
Nicolas
260e31c68b
Merge branch 'nsc/new-pricing'
2024-05-30 16:08:31 -07:00
Nicolas
aa8133ca7f
Update load-testing-example.ts
2024-05-30 16:07:14 -07:00
Nicolas
0c115c6181
Merge pull request #216 from mendableai/nsc/new-pricing
...
feat: New pricing/limits changes
2024-05-30 15:36:59 -07:00
