Nicolas
1a16378fe8
Merge pull request #234 from JakobStadlhuber/feat/webhook-self-hosted
...
Add support for Self-Hosted Webhook URL Usage and added project_id into the webhook payload
2024-06-05 10:25:05 -07:00
Nicolas
9640bf087e
Merge pull request #238 from mendableai/feat/better-gdrive-pdf-fetch
...
[Feat] Improved the scrape for gdrive pdfs
2024-06-05 10:20:46 -07:00
Nicolas
ff53db8c6d
Merge pull request #236 from JakobStadlhuber/doc/kubernetes
...
Add Kubernetes configuration for Firecrawl deployment
2024-06-05 10:20:32 -07:00
Nicolas
7cb14edec8
Nick:
2024-06-05 10:13:52 -07:00
Rafael Miller
9e000ded03
Merge branch 'main' into feat/better-gdrive-pdf-fetch
2024-06-05 14:07:56 -03:00
Nicolas
6d76037f6d
Merge pull request #239 from mendableai/feat/scroll-xpaths
...
[Feat] Added scroll xpaths on fire-engine for handling readme docs
2024-06-05 10:05:47 -07:00
rafaelsideguide
ccc55127d6
Added scroll xpaths on fire-engine for handling readme docs
2024-06-05 11:48:41 -03:00
rafaelsideguide
b5045d1661
[feat] improved the scrape for gdrive pdfs
2024-06-04 17:47:28 -03:00
Nicolas
a547f9a78e
Merge pull request #237 from mendableai/nsc/custom-vanta-refactor
...
Refactor custom scraping and added security center vanta
2024-06-04 12:28:30 -07:00
Nicolas
96257b7b17
Update handleCustomScraping.ts
2024-06-04 12:22:46 -07:00
Nicolas
de049a5ae1
Update README.md
2024-06-04 12:18:57 -07:00
Nicolas
73da34c887
Update README.md
2024-06-04 12:18:40 -07:00
Rafael Miller
f98cd75cd0
Merge pull request #140 from mendableai/calebpeffer-github-seo
...
Update README.md
2024-06-04 16:18:09 -03:00
Nicolas
674500affa
Nick:
2024-06-04 12:15:39 -07:00
Rafael Miller
f476e776c0
Merge pull request #213 from mattjoyce/github-workflows
...
Script to check local vs published versions of sdk packages
2024-06-04 16:12:17 -03:00
Jakob Stadlhuber
3cca2e3bfe
Update Docker image names in README
...
The Docker image names in README.md in the k8n example scripts were updated to ensure they accurately reflect the intended container images. This change primarily involves switching the names of the API and Playwright service Docker images.
2024-06-04 21:09:43 +02:00
rafaelsideguide
d54ec94bfd
lgtm. Added some docs and requirements.txt
2024-06-04 16:01:21 -03:00
Jakob Stadlhuber
07246d0e1c
Update README file in k8n directory
...
Removed a redundant list item and updated section title for deleting Firecrawl. The "Delete Firecrawl" section is now more concise and does not refer specifically to an environment, making it more generally applicable.
2024-06-04 20:59:04 +02:00
Jakob Stadlhuber
078d4c8d41
Add Kubernetes configuration for Firecrawl deployment
...
Added new files for setting up Firecrawl on a Kubernetes Cluster. The files include Kubernetes manifests for deploying API, worker, playwright service, and Redis with associated ConfigMap and Secret associated resources. Also, updated the self-host documentation to include instructions for Kubernetes deployment.
2024-06-04 20:52:08 +02:00
Nicolas
fc04d5b033
Merge pull request #235 from mendableai/feat/gdrive-pdfs
...
[Feat] Added custom scraping for google-drive pdf usecase
2024-06-04 11:31:53 -07:00
rafaelsideguide
5ae4d1caf5
Update single_url.ts
2024-06-04 15:28:09 -03:00
Jakob Stadlhuber
9e5ddec207
Remove default webhook URL from .env.example
...
The default value for the SELF_HOSTED_WEBHOOK_URL in the .env.example file was removed to prevent unintentional exposure or usage. The users are now required to explicitly specify
2024-06-04 19:56:35 +02:00
Jakob Stadlhuber
6208f4207d
Add support for Self-Hosted Webhook URL Usage and added project_id into the webhook payload
...
This commit introduces the capability of using a Self-Hosted Webhook URL. The application now checks for a self-hosted URL before querying the database for the webhook settings. If a Self-Hosted Webhook URL is set in the environment variables, it will be used directly, diminishing unnecessary database queries.
2024-06-04 19:55:07 +02:00
rafaelsideguide
93f3098672
build files
2024-06-04 14:54:54 -03:00
rafaelsideguide
64a4338ff0
Update single_url.ts
2024-06-04 14:40:05 -03:00
Rafael Miller
02fe470e20
Merge pull request #148 from mendableai/nsc/improvemnts-fixes-misc
...
Better fallbacks for initial crawl start
2024-06-04 14:31:10 -03:00
Rafael Miller
665a40d9f4
Merge pull request #212 from mendableai/bugfix/partial-data-js-sdk
...
[Bug] Improved js response and test for getting partial_data
2024-06-04 14:05:23 -03:00
rafaelsideguide
1f4c6b7a87
Update package.json
2024-06-04 13:59:48 -03:00
Rafael Miller
19c67916d4
Merge pull request #211 from mendableai/fix/rename-variables
...
[Fix] Changed timeout parameter name on js sdk
2024-06-04 13:57:58 -03:00
Rafael Miller
f4f87b5374
Merge branch 'main' into bugfix/partial-data-js-sdk
2024-06-04 13:40:42 -03:00
Rafael Miller
f17cb1a0d4
Merge pull request #224 from mattjoyce/playwright-service-bug-222
...
Playwright service bugs #222 #179 #197
2024-06-04 12:05:56 -03:00
rafaelsideguide
4e3a0495d7
updated version 0.0.12 -> 0.0.13
...
- [ ] publish
2024-06-04 12:03:55 -03:00
Rafael Miller
b80fb374e5
Merge branch 'main' into playwright-service-bug-222
2024-06-04 11:57:17 -03:00
rafaelsideguide
6920ec8a61
bugfixing. already on main
2024-06-04 11:05:50 -03:00
Nicolas
d6762386f8
Update fly-direct.yml
2024-06-04 01:09:04 -07:00
Nicolas
d10c0839b0
Update fly-direct.yml
2024-06-04 01:04:06 -07:00
Nicolas
5d50b259b7
Create fly-direct.yml
2024-06-04 00:42:07 -07:00
Nicolas
d91b725c6f
Update fly.toml
2024-06-04 00:41:15 -07:00
Nicolas
cbf8d79cce
Update pdfProcessor.ts
2024-06-04 00:13:37 -07:00
Nicolas
3fc9004ba8
Update fly.toml
2024-06-03 23:49:46 -07:00
Nicolas
0cc7031acb
Update fly.yml
2024-06-03 23:47:10 -07:00
Nicolas
2ea01f1456
Update single_url.ts
2024-06-03 23:42:39 -07:00
Nicolas
3563e3ae45
Update fly.yml
2024-06-03 23:34:52 -07:00
Nicolas
854d5b3cb3
Update single_url.ts
2024-06-03 23:32:55 -07:00
Nicolas
99059814a8
Nick:
2024-06-03 21:32:48 -07:00
Nicolas
918059ee9e
Merge branch 'main' into nsc/improvemnts-fixes-misc
2024-06-03 16:46:02 -07:00
Nicolas
93bb53271e
Merge branch 'nsc/improved-blocklist'
2024-06-03 16:44:33 -07:00
Nicolas
38e583f66c
Update socialBlockList.test.ts
2024-06-03 16:44:23 -07:00
Nicolas
b26c5f1588
Merge pull request #185 from mendableai/nsc/improved-blocklist
...
Improvements to the blocklist regex
2024-06-03 16:43:34 -07:00
Nicolas
c69c89f838
Nick:
2024-06-03 16:42:42 -07:00
