ToolJet/marketplace
namanmathur372 b6822a3ce2 Integrate Pinecone Plugin with Core Operations for Advanced Vector Management (#11188)
* Initialize ToolJet plugin with default files and installed dependencies

Created the folder structure and default files for the pinecone plugin
the server/src/assets/marketplace/plugins.json file also got updated automatically and added an entry for the Pinecone plugin.
Installed the specified versions of @grpc/grpc-js@^1.12.0 and @pinecone-database/pinecone@^3.0.3 packages into the project.

* Update Pinecone plugin manifest with API key authentication

Updated Pinecone plugin manifest to include API key authentication as an encrypted string.
Added API key field with description and help text for generating the key via the Pinecone Console.
Marked apiKey as a required field in the schema.

* Enhance Pinecone operations.json with vector management options

Amended operations.json to support various vector management operations for the Pinecone plugin.
Added new fields for index, operation, and specific properties for operations like get_index_stats, list_vector_ids, fetch_vectors, upsert_vectors, update_vector, and delete_vectors.
Introduced support for vector ID management, metadata, namespace, and filter options in JSON format.
Updated the schema type from api to database to better reflect the plugin's functionality.

* Add query operations and enhance types for Pinecone plugin

Created query_operations.ts to handle Pinecone operations: getIndexStats, listVectorIds, fetchVectors, upsertVectors, updateVector, and deleteVectors.
Added error handling, namespace support, and consolidated vector deletion logic.
Updated types.ts with detailed SourceOptions and QueryOptions, including fields like index, ids, vectors, and pagination.
Introduced types for Vector, SparseValues, and an enum for available operations, ensuring type safety.
Functions for getIndexStats and listVectorIds work correctly, but other operations are not functioning as expected.

* Implement Pinecone service with query operations and connection handling

Updated index.ts with PineconeService to handle operations: getIndexStats, listVectorIds, fetchVectors, upsertVectors, updateVector, and deleteVectors.
Added connection testing and API key handling functions.
Error handling for invalid operations included.
Only getIndexStats and listVectorIds are working correctly; other operations need debugging.

* Fix operations and add query vector operation

* Update placeholders for boolean fields

Set placeholders for delete_all, include_values, and include_metadata to indicate "true (false by default)" for clearer defaults.

---------

Co-authored-by: parthy007 <parthadhikari1812@gmail.com>
2024-11-08 12:21:01 +05:30
..
_templates/plugin/new Enhancements to the process of developing a Marketplace plugin (#5777) 2023-03-24 17:05:08 +05:30
plugins Integrate Pinecone Plugin with Core Operations for Advanced Vector Management (#11188) 2024-11-08 12:21:01 +05:30
scripts [chore] Eslint fixes (#5988) 2023-04-11 15:34:58 +05:30
.eslintignore fixes: integrating plugins from marketplace (#5678) 2023-03-16 18:17:25 +05:30
.eslintrc plugin: aws textract 2023-04-19 21:12:57 +05:30
.gitignore [chore] Eslint fixes (#5988) 2023-04-11 15:34:58 +05:30
lerna.json Enhancements to the process of developing a Marketplace plugin (#5777) 2023-03-24 17:05:08 +05:30
package-lock.json Integrate Pinecone Plugin with Core Operations for Advanced Vector Management (#11188) 2024-11-08 12:21:01 +05:30
package.json sync marketplace and tooljet database changes 2024-10-28 23:37:40 +05:30
README.md [Feature] Make plugins installable (#3069) 2022-10-27 16:59:43 +05:30
tsconfig.json feat: Adds portkey plugin (#10337) 2024-09-18 11:27:58 +05:30

Tooljet marketplace

Steps to install npm package to a plugin

npm i <npm-package-name> --workspace=<plugin-name-in-package-json>

Steps to build

npm install
npm run build --workspaces

Update the plugins to S3 bucket

AWS_ACCESS_KEY_ID=<key> SECRET_ACCESS_KEY=<secret> AWS_BUCKET=<bucket> node scripts/upload-to-s3.js