Commit graph

370 commits

Author SHA1 Message Date
Adish M
6ddbb7fedc
feat: implement SSRF protection with URL validation across plugins (#14257)
* feat: implement SSRF protection with URL validation across plugins

* refactor SSRF protection to focus on cloud metadata endpoints and improve configuration options

* remove legacy whitelist functionality and streamline SSRF validation process

* enhance SSRF protection by adding configurable blocked schemes and validation checks

* enhance SSRF protection by integrating configurable options across services

* replace dns.lookup with dns.lookup from dns module for improved clarity

* refactor: enhance SSRF protection by merging request options and improving IP format normalization

* Fix: update comments for clarity and enhance IP normalization in SSRF protection

* enhance SSRF protection by validating URL and applying protection options in GraphqlQueryService

* enhance SSRF protection with detailed validation for redirects and URL schemes
2026-02-23 17:25:13 +05:30
Akshay
8c9d79c012
Fix: Use loadSync for gRPC filesystem proto loading and add lightweight service discovery (#15225)
* fix(grpcv2): use loadSync for filesystem proto loading to prevent server crash

protobufjs has an unfixed bug (protobufjs/protobuf.js#1098) where
async Root.load() calls resolveAll() outside its try-catch in the
finish() callback. When resolveAll() throws (e.g. unresolvable types),
the error becomes an uncaught exception that crashes the Node.js
process — the Promise never resolves/rejects.

Switch from protoLoader.load() to protoLoader.loadSync() for all
filesystem-based proto loading. With loadSync, resolveAll() errors
propagate as normal synchronous throws caught by existing try-catch
blocks. This is consistent with loadProtoFromRemoteUrl() which
already uses loadSync.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(grpcv2): add filesystem proto discovery with lightweight scanning

Add discoverServiceNames and discoverMethodsForServices to support
two-phase service discovery from filesystem proto files. Uses
protobufjs.parse() for lightweight name scanning (~30KB/file) and
only loads full gRPC definitions for selected services, preventing
OOM on large proto directories.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(DynamicSelector): add multi-select and autoFetch for grpcv2 filesystem services

Add isMulti and autoFetch props to DynamicSelector. autoFetch triggers
service discovery on mount without requiring a manual button click,
and skips cache persistence to avoid false "Unsaved Changes" prompts.
Multi-select renders services as chips with custom styles.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(data-sources): handle non-array elements in resolveKeyValuePair

resolveKeyValuePair assumed all array option elements are sub-arrays
(like metadata key-value pairs). Options like selected_services contain
plain strings, causing arr.map crash during test connection. Guard with
Array.isArray check and fall back to resolveValue for scalar elements.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(grpcv2): require service selection for filesystem mode in query manager

Filesystem mode without selected services would fall back to full
proto discovery (loading every file), defeating the purpose of the
two-phase discovery. Now shows an error asking the user to select
services in the datasource config instead.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(grpcv2): require selected services for filesystem test connection

Test connection in filesystem mode now validates that at least one
service is selected and uses a selected service for the connectivity
check instead of picking an arbitrary one from the proto directory.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(grpcv2): simplify filesystem test connection to validate proto parsing

Test connection for filesystem mode now just validates that proto files
can be parsed and services discovered — no service selection required.
Removes the meaningless waitForReady check which only tested TCP
connectivity without validating anything about the proto definitions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(grpcv2): remove filesystem branch from discoverServices

Filesystem mode never flows through discoverServices — it uses the
two-phase discoverServiceNames + discoverMethodsForServices path.
Remove the dead branch and add a comment documenting the filesystem
flow for clarity.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(grpcv2): add comment documenting test connection behavior per mode

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(grpcv2): add TCP connectivity check for filesystem mode test connection

Filesystem mode now falls through to checkFirstServiceConnection (waitForReady)
like reflection and URL modes, instead of returning early after proto parsing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* cleanup(DynamicForm): remove unused snake_case prop aliases for isMulti and autoFetch

No plugin manifest uses is_multi or auto_fetch — the gRPC v2 manifest
(the only consumer of these props) uses camelCase exclusively, and there
is no transformation layer in the pipeline.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(DynamicSelector): suppress noAccessError flash during loading

The no-access warning and red border briefly flashed on page reload
because validateSelectedValue ran with an empty array before the
fetch completed. Gate both on !isLoading so they only appear after
data is actually loaded.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(DynamicSelector): skip cache validation for autoFetch on unrelated prop changes

When autoFetch is enabled, cache is never persisted to options (to avoid
"Unsaved Changes"). So every time selectedDataSource changes for
unrelated reasons (e.g. title edit), the cache-checking useEffect finds
no cache and calls validateSelectedValue([]), falsely triggering the
no-access warning. Skip this effect for autoFetch since it has its own
dedicated fetch/validation lifecycle.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(grpcv2): remove QueryResult wrapping from plugin invokeMethod returns

grpcv2 plugin methods (discoverServiceNames, discoverMethodsForServices)
were returning QueryResult-wrapped responses which got double-wrapped by
DataSourcesService.invokeMethod, causing GRPCv2Component to crash with
"servicesData.services.map is not a function" when opening filesystem
proto queries.

Plugin invokeMethod now returns raw data (arrays) instead of QueryResult
objects. The server's invokeMethod always wraps with { status: 'ok', data }
consistently. DynamicSelector adds an Array.isArray guard for plugins
that return raw arrays vs { data: [...] }.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(DynamicSelector): skip access validation for autoFetch fields

autoFetch fields (e.g. gRPC services) are discovered from proto files,
not OAuth-scoped resources — "no access" warnings don't apply.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(grpcv2): unify service discovery into single getServiceDefinitions entry point

Consolidate discoverServices, discoverServiceNames, and discoverMethodsForServices
into two clear methods: listServices (lightweight name scan for DS config) and
getServiceDefinitions (full method discovery for query editor, all modes).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(grpcv2): prevent OOM on filesystem test connection and query fallback

Filesystem test connection no longer parses proto files — just counts
them with fast-glob and checks TCP connectivity via a raw gRPC client.
Query execution fallback after server restart now uses the lightweight
protobufjs.parse() scanner instead of the heavy protoLoader.loadSync()
path. Removes two dead functions (discoverServicesIndividually,
discoverServicesUsingFilesystem).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update version to 3.20.95-lts across all components

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: gsmithun4 <gsmithun4@gmail.com>
2026-02-17 17:52:48 +05:30
abhijeet760
af09402477
fixed in operation json (#15206) 2026-02-09 22:44:47 +05:30
abhijeet760
3ef6e75ce2
document instead of documents (#15201) 2026-02-09 15:02:19 +05:30
Midhun G S
c8fe500a1d
Migrated aws-sdk v2 to v3 (#15023)
* chore: update bcrypt to version 6.0.0 in package.json

* chore: remove unused devDependencies from package.json

* Refactor code structure for improved readability and maintainability

* Refactor code structure for improved readability and maintainability
2026-02-04 17:27:17 +05:30
gsmithun4
f8daa74171 rebase 2026-01-30 14:40:54 +05:30
Midhun G S
ef998eed20
chore: package upgrades (#15009)
* chore: update bcrypt to version 6.0.0 in package.json

* chore: remove unused devDependencies from package.json

* Implement feature X to enhance user experience and optimize performance
2026-01-22 11:34:05 +05:30
Akshay
25261d70f2
Refactor: gRPC safe proto load (#14233)
* refactor: failsafe gRPC service discovery with file loads

* refactor: search only service names

---------

Co-authored-by: gsmithun4 <gsmithun4@gmail.com>
2026-01-19 21:16:16 +05:30
abhijeet760
2e8cced91e
added none based switch case (#14954) 2026-01-19 19:49:47 +05:30
Midhun G S
f68c8e1885
[Vulnerability] package upgrades (#14861)
* chore: update nestjs-otel to version 8.0.1 and add systeminformation override

* remove unused API metrics configuration from OpenTelemetry setup

* fixed ldap flaky case

* Update: Sendgrid package to 8.1.6 (#14833)

* updated sendgrid

* fix: update listBuckets function to use default options for ListBucketsCommand

---------

Co-authored-by: gsmithun4 <gsmithun4@gmail.com>

* Vulnerability Issues (#14834)

* Security Fixes

* set version

* utl harper db

* s3

* marketplace package-lock

* Remove package-lock.json files

* chore: update lerna to version 9.0.3 in package.json

* Implement code changes to enhance functionality and improve performance

* removing options from s3 bucket in markeplace

* supabase build type fixes

---------

Co-authored-by: Adish M <adish.madhu@gmail.com>
Co-authored-by: Yukti Goyal <yuktigoyal02@gmail.com>
Co-authored-by: Adish M <44204658+adishM98@users.noreply.github.com>
Co-authored-by: abhijeet760 <abhijeet@tooljet.com>
Co-authored-by: Rudhra Deep Biswas <98055396+rudeUltra@users.noreply.github.com>
Co-authored-by: Rudhra Deep Biswas <rudra21ultra@gmail.com>
2026-01-08 17:46:51 +05:30
abhijeet760
a0de33c5eb lint errors 2026-01-02 07:53:37 +00:00
abhijeet760
e7a7fbcf49 whitespace change 2025-12-26 13:05:28 +00:00
abhijeet760
19b824e2fa whitespace changes 2025-12-26 12:59:28 +00:00
abhijeet760
3055059063 added list tables for mongo and postgres 2025-12-26 12:10:00 +00:00
gsmithun4
0511d820df Implement feature X to enhance user experience and optimize performance 2025-12-10 18:51:56 +05:30
abhijeet760
b06e9b2b04
Enhance : ClickHouse module updated (#14666)
* updated clickhouse npm

* added_parser_to__tooljet

* updated response structure and removed data migration file

* resolved comments

* removed the fields from manifest.json

* improved error message
2025-12-10 15:16:03 +05:30
Pratush Sinha
861e717248
MongoDB connection with srv support (#14626)
* MongoDB connection with srv support

* design changes and migration

* changes in design

* fix password field is coming disabled.

* change with save button

* changes in the save button and encrypt

* Changes for mongoDb SRV after Code Review

* migration script

* Changes in the UI

* fixed the db issue in connection

* changes comment removed

* SSl issue resolved

* Review changes made

* editing connection string affects only individual fields

* fixed dataSource Saving in imported apps

* fixed imported apps ds save

---------

Co-authored-by: Pratush <pratush@Pratushs-MacBook-Pro.local>
2025-12-10 15:06:10 +05:30
Pratush Sinha
c602b2ccb4
Hotfix: MySQL should reject unauthorised connection only when certificates are provided (#14710)
* Hotfix ] MySQL - Reject Unauthorised SSL connection

* fixed sourceOptions.ssl_certificate ?? 'none'

---------

Co-authored-by: Pratush <pratush@Pratushs-MacBook-Pro.local>
Co-authored-by: Ganesh Kumar <ganesh8056234@gmail.com>
2025-12-05 19:15:47 +05:30
Midhun G S
07e8d3a3a3
Merge pull request #14634 from ToolJet/release-lts/marketplace-sprint-14
Marketplace release : Sprint 14
2025-11-28 17:50:25 +05:30
Pratush Sinha
4ebd30ece7
Added option to send Sender's Name (#14650)
* Added option to send Sender's Name

* change made to Sender Name

* bump version to 3.20.46-lts in all relevant files

---------

Co-authored-by: Pratush <pratush@Pratushs-MacBook-Pro.local>
Co-authored-by: gsmithun4 <gsmithun4@gmail.com>
2025-11-27 21:49:25 +05:30
Ganesh Kumar
3a4d7da1c1
Feat: Query level timeout for Postgresql datasource (#14612)
* POstgresql query level timeout implementation

* for query without parameters query level timeout has been updated

* Query level timeout at Run Query endpoint

* label for timeout is updated

* Timeout is applied at run query level

* Method name has been modified

* ref updated

* Error message modified

* Abort controller is abstracted

---------

Co-authored-by: gsmithun4 <gsmithun4@gmail.com>
2025-11-27 19:47:59 +05:30
Pratush Sinha
368a73176c
Fix: MySQL SSL Toggle reject unauthorised connection (#14551)
* SSl Toggle Issue

* review comments resolved

---------

Co-authored-by: Pratush <pratush@Pratushs-MacBook-Pro.local>
2025-11-24 11:03:06 +05:30
Ganesh Kumar
6e5badccc3
Hotfix: Support for specifying OAuth scopes for BigQuery connections (#14567) 2025-11-13 21:40:02 +05:30
parthy007
d1f62405a4 fix: configure body correctly for token call
fix: add hubspot in querymanager auth flow
2025-10-23 15:44:51 +05:30
Parth
249d7a1135
Feature: Validate and save datasource OAuth creds in multi-tenant mode (#14043)
* fix: change schema to hide save button & add default value

* enhance: Show connect button in UI at data-config

* fix: extract source_options correctly

enhance: add userID in cache logic for snowflake

* enhance: add backend logic to save tokens correctly

fix: change IDataSourcesUtilService interface

* fix: add read docs button for oauth plugins

fix: shrink the connect button

* fix: remove extra code and fix comments

* fix: generate cache key correctly
2025-10-22 17:19:46 +05:30
Parth
03505b0174
fix: token headers & client-auth dropdown (#14082) 2025-10-22 15:19:20 +05:30
Ganesh Kumar
ac5a65179d Merge branch 'lts-3.16' into release-lts/marketplace-sprint-13 2025-10-22 14:12:14 +05:30
Priyansh Rai
beb8bcfc65
Fix: Capitalisation inconsistencies in labels (#14174)
* fixed capitalization issue

Signed-off-by: thesynthax <kushagra1403@gmail.com>

* updated changes

Signed-off-by: thesynthax <kushagra1403@gmail.com>

* fix: standardize label casing and formatting in plugin manifests

* fix: standardize label casing in plugin manifests

---------

Signed-off-by: thesynthax <kushagra1403@gmail.com>
Co-authored-by: thesynthax <kushagra1403@gmail.com>
2025-10-22 13:59:18 +05:30
Ajith KV
7d28810652
Add cypress test cases for enterprise user groups and permissions (#14340)
* Add cypress test cases for enterprise user groups and permissions

* update constants

* update user groups test cases

* update spec

* update groups cases

* update lock.json

* update custom group cases

* update docker file

* update first user onboarding and platform commands

* add case for UI  promote and release permission
2025-10-17 12:38:44 +05:30
Pratush Sinha
e8f51f91cd
Fix: Error handling in Elastic Search test connection (#14308)
* error Handeling in Elastic Search Test Connection

* added requestTimeout
2025-10-13 16:23:58 +05:30
Parth
2db7909a7c
fix: decode the base64 string (#14138) 2025-10-10 13:21:47 +05:30
Ganesh Kumar
35a343d813
Hotfix: MariaDB datasource connection handling (#14277)
* Mariadb graceful connection handling

* fixed review comments
2025-10-08 14:37:43 +05:30
Ganesh Kumar
0f00162fcc
Feat: Support for service account authentication in google sheet datasource (#14209)
* working on google sheet service account flow

* service account flow logic added

* for service account flow skip the Oauth error flow

* added datamigration to backfill the authentication option for GoogleSheets
2025-10-08 14:34:05 +05:30
Ganesh Kumar
69bb50ad48
Feature : AWS Default Credential Provider Chain as an authentication option in AWS S3 datasource (#14171)
* added default credential chain flow

* Bump version to 3.20.5-lts across all components and add tag-and-push script for releases

---------

Co-authored-by: gsmithun4 <gsmithun4@gmail.com>
2025-09-26 20:48:25 +05:30
Parth
9167b950ae
Fix: Snowflake input rendering (ClientId & Client-secret) in cloud & OAuth plugin failure due to undefined environmentId (#14167)
* fix: use selectedEnvironment instead of currentAppEnvironmentId

* fix: clientid and secret rendering
2025-09-26 20:45:13 +05:30
Akshay
ca2e70de82
Fix: gRPC reflection client and adds fuzzy search for gRPC service (#14067)
* refactor: use service descriptor retrieval and pass callOptions

* refactor: remove redundant comments

* refactor:  use sdk methods to get appropriate proto definition on reflection

* fix: changes not save immediately

* fix: improve loading state handling in GRPCv2 component

* feat: add filesystem support for loading proto files

* feat: implement fuzzy search and debounce functionality
2025-09-26 20:30:02 +05:30
gsmithun4
f40be5b1b5 feat(logging): integrate transaction logging across various guards and services
- Updated `AbilityGuard` to utilize `TransactionLogger` for logging execution time and errors.
- Enhanced `ResponseInterceptor` to include transaction metadata in logs.
- Modified `QueryAuthGuard`, `ValidateQueryAppGuard`, and `ValidateQuerySourceGuard` to log completion times and transaction IDs.
- Introduced `TransactionLogger` service for structured logging with transaction context.
- Added transaction ID and route information to request context in `RequestContextMiddleware`.
- Updated `JwtStrategy` to log validation completion times.
- Refactored logging configuration in `AppModuleLoader` to support pretty printing in non-production environments.
- Removed console logs in favor of structured logging for better traceability.
2025-09-19 02:08:49 +05:30
Akshay Sasidharan
88fa169af8 refactor: make code more strict in typing 2025-09-12 14:23:19 +05:30
Akshay Sasidharan
d44f755df9 refactor: revise function names for clarity 2025-09-12 11:26:29 +05:30
Akshay Sasidharan
1026593b40 fix: send metadata not in call options 2025-09-12 02:18:15 +05:30
Ganesh Kumar
abf26d6439 Merge branch 'release-lts/marketplace-plugin-set6' of https://github.com/tooljet/tooljet into release-lts/marketplace-plugin-set6 2025-09-10 13:54:26 +05:30
Ganesh Kumar
2e069f5af8 Merge branch 'lts-3.16' into release-lts/marketplace-plugin-set6 2025-09-10 13:54:02 +05:30
Akshay Sasidharan
e234c5a66f refactor: remove deadcode 2025-09-09 12:19:37 +05:30
Akshay
186d9db5f5
Apply suggestion from @Copilot
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-09-08 23:37:12 +05:30
Akshay Sasidharan
3d61aad3e9 refactor: simplify non-TLS execution by consolidating metadata handling 2025-09-08 23:16:03 +05:30
Akshay Sasidharan
37a308c9a7 refactor: streamline gRPC call execution and metadata handling 2025-09-08 23:07:40 +05:30
Akshay Sasidharan
e61b90ac64 fix: remove lowercase conversion 2025-09-08 20:35:58 +05:30
Akshay Sasidharan
509afde5e5 refactor: enhance gRPC call handling with unified call options for metadata 2025-09-08 19:43:37 +05:30
Priyansh Rai
96cc85943b
fix: correct placeholder and description text in postgresql, notion, and mssql plugins operations schemas (#13643) 2025-09-08 15:01:10 +05:30
Priyansh Rai
ea656b422f
fix(cosmosdb): add partition key placeholder in cosmosdb operations (#13642) 2025-09-08 14:51:37 +05:30