mirror of
https://github.com/ToolJet/ToolJet
synced 2026-04-29 09:27:32 +00:00
3 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Akshay
|
8c9d79c012
|
Fix: Use loadSync for gRPC filesystem proto loading and add lightweight service discovery (#15225)
* fix(grpcv2): use loadSync for filesystem proto loading to prevent server crash protobufjs has an unfixed bug (protobufjs/protobuf.js#1098) where async Root.load() calls resolveAll() outside its try-catch in the finish() callback. When resolveAll() throws (e.g. unresolvable types), the error becomes an uncaught exception that crashes the Node.js process — the Promise never resolves/rejects. Switch from protoLoader.load() to protoLoader.loadSync() for all filesystem-based proto loading. With loadSync, resolveAll() errors propagate as normal synchronous throws caught by existing try-catch blocks. This is consistent with loadProtoFromRemoteUrl() which already uses loadSync. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(grpcv2): add filesystem proto discovery with lightweight scanning Add discoverServiceNames and discoverMethodsForServices to support two-phase service discovery from filesystem proto files. Uses protobufjs.parse() for lightweight name scanning (~30KB/file) and only loads full gRPC definitions for selected services, preventing OOM on large proto directories. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(DynamicSelector): add multi-select and autoFetch for grpcv2 filesystem services Add isMulti and autoFetch props to DynamicSelector. autoFetch triggers service discovery on mount without requiring a manual button click, and skips cache persistence to avoid false "Unsaved Changes" prompts. Multi-select renders services as chips with custom styles. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(data-sources): handle non-array elements in resolveKeyValuePair resolveKeyValuePair assumed all array option elements are sub-arrays (like metadata key-value pairs). Options like selected_services contain plain strings, causing arr.map crash during test connection. Guard with Array.isArray check and fall back to resolveValue for scalar elements. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(grpcv2): require service selection for filesystem mode in query manager Filesystem mode without selected services would fall back to full proto discovery (loading every file), defeating the purpose of the two-phase discovery. Now shows an error asking the user to select services in the datasource config instead. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(grpcv2): require selected services for filesystem test connection Test connection in filesystem mode now validates that at least one service is selected and uses a selected service for the connectivity check instead of picking an arbitrary one from the proto directory. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(grpcv2): simplify filesystem test connection to validate proto parsing Test connection for filesystem mode now just validates that proto files can be parsed and services discovered — no service selection required. Removes the meaningless waitForReady check which only tested TCP connectivity without validating anything about the proto definitions. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(grpcv2): remove filesystem branch from discoverServices Filesystem mode never flows through discoverServices — it uses the two-phase discoverServiceNames + discoverMethodsForServices path. Remove the dead branch and add a comment documenting the filesystem flow for clarity. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(grpcv2): add comment documenting test connection behavior per mode Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(grpcv2): add TCP connectivity check for filesystem mode test connection Filesystem mode now falls through to checkFirstServiceConnection (waitForReady) like reflection and URL modes, instead of returning early after proto parsing. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * cleanup(DynamicForm): remove unused snake_case prop aliases for isMulti and autoFetch No plugin manifest uses is_multi or auto_fetch — the gRPC v2 manifest (the only consumer of these props) uses camelCase exclusively, and there is no transformation layer in the pipeline. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(DynamicSelector): suppress noAccessError flash during loading The no-access warning and red border briefly flashed on page reload because validateSelectedValue ran with an empty array before the fetch completed. Gate both on !isLoading so they only appear after data is actually loaded. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(DynamicSelector): skip cache validation for autoFetch on unrelated prop changes When autoFetch is enabled, cache is never persisted to options (to avoid "Unsaved Changes"). So every time selectedDataSource changes for unrelated reasons (e.g. title edit), the cache-checking useEffect finds no cache and calls validateSelectedValue([]), falsely triggering the no-access warning. Skip this effect for autoFetch since it has its own dedicated fetch/validation lifecycle. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(grpcv2): remove QueryResult wrapping from plugin invokeMethod returns grpcv2 plugin methods (discoverServiceNames, discoverMethodsForServices) were returning QueryResult-wrapped responses which got double-wrapped by DataSourcesService.invokeMethod, causing GRPCv2Component to crash with "servicesData.services.map is not a function" when opening filesystem proto queries. Plugin invokeMethod now returns raw data (arrays) instead of QueryResult objects. The server's invokeMethod always wraps with { status: 'ok', data } consistently. DynamicSelector adds an Array.isArray guard for plugins that return raw arrays vs { data: [...] }. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(DynamicSelector): skip access validation for autoFetch fields autoFetch fields (e.g. gRPC services) are discovered from proto files, not OAuth-scoped resources — "no access" warnings don't apply. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(grpcv2): unify service discovery into single getServiceDefinitions entry point Consolidate discoverServices, discoverServiceNames, and discoverMethodsForServices into two clear methods: listServices (lightweight name scan for DS config) and getServiceDefinitions (full method discovery for query editor, all modes). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(grpcv2): prevent OOM on filesystem test connection and query fallback Filesystem test connection no longer parses proto files — just counts them with fast-glob and checks TCP connectivity via a raw gRPC client. Query execution fallback after server restart now uses the lightweight protobufjs.parse() scanner instead of the heavy protoLoader.loadSync() path. Removes two dead functions (discoverServicesIndividually, discoverServicesUsingFilesystem). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: update version to 3.20.95-lts across all components --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: gsmithun4 <gsmithun4@gmail.com> |
||
|
Akshay
|
ca2e70de82
|
Fix: gRPC reflection client and adds fuzzy search for gRPC service (#14067)
* refactor: use service descriptor retrieval and pass callOptions * refactor: remove redundant comments * refactor: use sdk methods to get appropriate proto definition on reflection * fix: changes not save immediately * fix: improve loading state handling in GRPCv2 component * feat: add filesystem support for loading proto files * feat: implement fuzzy search and debounce functionality |
||
|
Akshay
|
ca71355d8e
|
Feature[LTS]: GRPC Plugin revamp (#13894)
* feat: Add new gRPC v2 plugin - Enhanced DataSourcesController to support invoking methods on data sources. - Introduced InvokeDataSourceMethodDto for method invocation requests. - Added WhitelistPluginGuard to restrict method invocation based on data source kind. - Updated IDataSourcesController and IDataSourcesService interfaces to include invoke method functionality. - Implemented invokeMethod in DataSourcesService to handle method calls for gRPC v2. - Added method whitelisting for gRPC v2 in the service layer. * refactor: invokeMethod to use method dispatch pattern * fix: improve error handling in testConnection method * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * refactor: update types and interface * fix: remove unnecessary border styles for improved UI consistency * refactor: simplify error handling - Introduced a new SCSS file for styling the gRPC v2 query editor components. - Styled request sections, server URL input, dropdown menus, and method buttons for better user experience. refactor(grpcv2): enhance error handling and type safety - Updated error handling in gRPC query service to use a new `toError` utility function for consistent error conversion. - Improved type definitions for gRPC client and service methods to ensure better type safety. - Refactored method existence validation and gRPC call execution to handle errors more gracefully. fix(grpcv2): improve service discovery and proto file loading - Enhanced service discovery logic to handle reflection and proto file clients more robustly. - Added checks for valid service constructors and improved error messages for better debugging. - Updated the `loadProtoFromRemoteUrl` function to handle HTTP errors more effectively. chore(grpcv2): add utility functions for type checks - Introduced utility functions `isRecord`, `hasProperty`, and `toError` for better type checking and error handling throughout the codebase. * feat: add LegacyBanner component for gRPC legacy tag display * fix: styling changes * fix: correct text in LegacyBanner component * feat: enhance GRPCv2 component with request handling and state management improvements * refactor: update metadata handling in GRPCv2 component and service * refactor: update GRPCv2 component to use raw_message instead of requestData * fix: Styling issues and labels * refactor: gRPC test connection checks for proto file and url * fix: improve error handling in service discovery --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |