ToolJet/plugins
Akshay 8c9d79c012
Fix: Use loadSync for gRPC filesystem proto loading and add lightweight service discovery (#15225)
* fix(grpcv2): use loadSync for filesystem proto loading to prevent server crash

protobufjs has an unfixed bug (protobufjs/protobuf.js#1098) where
async Root.load() calls resolveAll() outside its try-catch in the
finish() callback. When resolveAll() throws (e.g. unresolvable types),
the error becomes an uncaught exception that crashes the Node.js
process — the Promise never resolves/rejects.

Switch from protoLoader.load() to protoLoader.loadSync() for all
filesystem-based proto loading. With loadSync, resolveAll() errors
propagate as normal synchronous throws caught by existing try-catch
blocks. This is consistent with loadProtoFromRemoteUrl() which
already uses loadSync.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(grpcv2): add filesystem proto discovery with lightweight scanning

Add discoverServiceNames and discoverMethodsForServices to support
two-phase service discovery from filesystem proto files. Uses
protobufjs.parse() for lightweight name scanning (~30KB/file) and
only loads full gRPC definitions for selected services, preventing
OOM on large proto directories.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(DynamicSelector): add multi-select and autoFetch for grpcv2 filesystem services

Add isMulti and autoFetch props to DynamicSelector. autoFetch triggers
service discovery on mount without requiring a manual button click,
and skips cache persistence to avoid false "Unsaved Changes" prompts.
Multi-select renders services as chips with custom styles.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(data-sources): handle non-array elements in resolveKeyValuePair

resolveKeyValuePair assumed all array option elements are sub-arrays
(like metadata key-value pairs). Options like selected_services contain
plain strings, causing arr.map crash during test connection. Guard with
Array.isArray check and fall back to resolveValue for scalar elements.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(grpcv2): require service selection for filesystem mode in query manager

Filesystem mode without selected services would fall back to full
proto discovery (loading every file), defeating the purpose of the
two-phase discovery. Now shows an error asking the user to select
services in the datasource config instead.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(grpcv2): require selected services for filesystem test connection

Test connection in filesystem mode now validates that at least one
service is selected and uses a selected service for the connectivity
check instead of picking an arbitrary one from the proto directory.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(grpcv2): simplify filesystem test connection to validate proto parsing

Test connection for filesystem mode now just validates that proto files
can be parsed and services discovered — no service selection required.
Removes the meaningless waitForReady check which only tested TCP
connectivity without validating anything about the proto definitions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(grpcv2): remove filesystem branch from discoverServices

Filesystem mode never flows through discoverServices — it uses the
two-phase discoverServiceNames + discoverMethodsForServices path.
Remove the dead branch and add a comment documenting the filesystem
flow for clarity.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(grpcv2): add comment documenting test connection behavior per mode

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(grpcv2): add TCP connectivity check for filesystem mode test connection

Filesystem mode now falls through to checkFirstServiceConnection (waitForReady)
like reflection and URL modes, instead of returning early after proto parsing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* cleanup(DynamicForm): remove unused snake_case prop aliases for isMulti and autoFetch

No plugin manifest uses is_multi or auto_fetch — the gRPC v2 manifest
(the only consumer of these props) uses camelCase exclusively, and there
is no transformation layer in the pipeline.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(DynamicSelector): suppress noAccessError flash during loading

The no-access warning and red border briefly flashed on page reload
because validateSelectedValue ran with an empty array before the
fetch completed. Gate both on !isLoading so they only appear after
data is actually loaded.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(DynamicSelector): skip cache validation for autoFetch on unrelated prop changes

When autoFetch is enabled, cache is never persisted to options (to avoid
"Unsaved Changes"). So every time selectedDataSource changes for
unrelated reasons (e.g. title edit), the cache-checking useEffect finds
no cache and calls validateSelectedValue([]), falsely triggering the
no-access warning. Skip this effect for autoFetch since it has its own
dedicated fetch/validation lifecycle.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(grpcv2): remove QueryResult wrapping from plugin invokeMethod returns

grpcv2 plugin methods (discoverServiceNames, discoverMethodsForServices)
were returning QueryResult-wrapped responses which got double-wrapped by
DataSourcesService.invokeMethod, causing GRPCv2Component to crash with
"servicesData.services.map is not a function" when opening filesystem
proto queries.

Plugin invokeMethod now returns raw data (arrays) instead of QueryResult
objects. The server's invokeMethod always wraps with { status: 'ok', data }
consistently. DynamicSelector adds an Array.isArray guard for plugins
that return raw arrays vs { data: [...] }.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(DynamicSelector): skip access validation for autoFetch fields

autoFetch fields (e.g. gRPC services) are discovered from proto files,
not OAuth-scoped resources — "no access" warnings don't apply.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(grpcv2): unify service discovery into single getServiceDefinitions entry point

Consolidate discoverServices, discoverServiceNames, and discoverMethodsForServices
into two clear methods: listServices (lightweight name scan for DS config) and
getServiceDefinitions (full method discovery for query editor, all modes).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(grpcv2): prevent OOM on filesystem test connection and query fallback

Filesystem test connection no longer parses proto files — just counts
them with fast-glob and checks TCP connectivity via a raw gRPC client.
Query execution fallback after server restart now uses the lightweight
protobufjs.parse() scanner instead of the heavy protoLoader.loadSync()
path. Removes two dead functions (discoverServicesIndividually,
discoverServicesUsingFilesystem).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update version to 3.20.95-lts across all components

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: gsmithun4 <gsmithun4@gmail.com>
2026-02-17 17:52:48 +05:30
..
_templates/plugin/new Feature/2395 - add eslint to plugins (#2402) 2022-03-10 12:29:48 +05:30
packages Fix: Use loadSync for gRPC filesystem proto loading and add lightweight service discovery (#15225) 2026-02-17 17:52:48 +05:30
schemas Add json schema for vscode autocomplete (#1962) 2022-01-27 10:55:16 +05:30
.eslintignore [chore] Eslint fixes (#5988) 2023-04-11 15:34:58 +05:30
.eslintrc.js [chore] Eslint fixes (#5988) 2023-04-11 15:34:58 +05:30
.gitignore Move plugins to root (#1728) 2022-01-17 12:38:17 +05:30
build-packages.js Revert "ignore building grpc plugin for cloud" 2023-05-02 18:28:44 +05:30
clean-packages.js Added configuration to select plugins (#3024) 2022-05-20 19:32:46 +05:30
create-client-entry.js Added configuration to select plugins (#3024) 2022-05-20 19:32:46 +05:30
create-operations-file.js Woocommerce integration 🚀 (#2866) 2022-05-19 11:38:14 +05:30
create-server-entry.js Added configuration to select plugins (#3024) 2022-05-20 19:32:46 +05:30
jest.config.js Feature/2395 - add eslint to plugins (#2402) 2022-03-10 12:29:48 +05:30
lerna.json Move plugins to root (#1728) 2022-01-17 12:38:17 +05:30
package-lock.json Migrated aws-sdk v2 to v3 (#15023) 2026-02-04 17:27:17 +05:30
package.json chore: package upgrades (#15009) 2026-01-22 11:34:05 +05:30
README.md fixed issue #7097 (#7099) 2023-08-04 08:27:38 +05:30
tsconfig.json [Unit test] Plugins - Googlesheet update operation (#2450) 2022-03-08 11:53:47 +05:30