mirror of
https://github.com/ToolJet/ToolJet
synced 2026-05-23 00:48:25 +00:00
* statement timeout for postgresql mssql mysql can now be configured from env * Enhance: Add clear all, cancel & apply functionality to TJDB filter-popup (#2344) * Update filter inputs * Update filter body * Style fixes * Handle invalid filters * Fix bugs * Feature: Add list all sheets and create new spreadsheet operation to GoogleSheets (#2370) * Add create spreadsheet functionality * Add list all sheets functionality * Updated delete and batch-update methods * Change interface to type * Enhance: Add read documentation link with data source drop down in query builder (#2162) * Add read-documentation link * Change copywriting for datasources name * Update new component for link * Increase input width Update redshift link * Fix the datasource name issue * Enhance: Improve error handling in Google-Sheets run query (#2286) * Add error details * Display error for incorrect JSON * Throw error for empty spreadsheetid and operator * Enhance: Error handling for PgSql, MySql and MsSql (#2389) * Enhance error handling for pgsql * Enhance error handling for mysql * Remove console logs * Enhance error handling for MsSql * Refactor error handling for consistency * Enhance: Rest api body to accept raw input instead of raw json. (#2249) * Enhanced rest api body to accept raw input instead of raw json. * Changed content type from application/json to text/json and changed copywrite to RAW. * Changed rest api body toggle label from 'RAW' to 'Raw'. * Added request label for static REST API data source. * Fixed issue where GET query failed since body was undefined. * Integrated json_body to add backward compatibility. * Removed console logs. * Added support for 'text/json' type in checkIfContentTypeIsJson function. * Made changes according to new frontend architecture in v3. * Fixed request URL field overflow issue. --------- Co-authored-by: Akshay Sasidharan <akshaysasidharan93@gmail.com> * Feature: Add SSL support to MongoDB datasource (#2430) * Add TLS support inputs in frontend * Add backend logic for TLS support * Add TLS inputs types * Update TLS label * Change ssl_certificate to tls_certificate * Update the file handling in tls * Update connection logic * Fix unlinking file issue * Remove catch block for unlinking file * Handle tls certs directly * Feature: Stripe plugin UI fixes with OpenAPI endpoint as source (#2725) * Fixed GET and DELETE request input renders and added all UX fixes (#9498) * fixed get and delete request input renders and added all UX fixes * Extracted Stripe plugin component as a separate component for Dynamic Form. * Resolved PR review comments and fixed issue where rendering path input field crashed the app. * Changed param name underline to dashed for tooltip and revamped input field clear button * Fixed a few sonarlint issues. * Removed the duplicate code by creating a separate function named RenderParameterFields. * Refactored computeOperationSelectionOptions function to not nest functions more than 4 levels deep. * Refactored RenderParameterFields function to reduce its Cognitive Complexity. * Made span tag with the 'button' interactive role focusable. * Inside switch case for codehinter in DynamicForm > getElementProps, Extracted nested ternary operation for theme into an independent statement. * Added keyboard listener to the clear button. * Removed opacity from select dropdown and operation from operation select dropdown UI. * Fixed syntax error in clearButton function. * Removed the package @nrwl/nx-linux-x64-gnu from marketplace dependencies. --------- Co-authored-by: Mansukh Kaur <mansukhkaur@Mansukhs-MacBook-Pro.local> Co-authored-by: Devanshu Rastogi <devanshu.rastogi05@gmail.com> * Made changes according to new app builder architecture in v3. * Fixed import issue for codehintor * Fixed issue where due to incorrect value, stripe queries crashed the entire app. --------- Co-authored-by: Mansukh Kaur <mansukhkaur@Mansukhs-MacBook-Pro.local> Co-authored-by: Devanshu Rastogi <devanshu.rastogi05@gmail.com> * Enhance: Improve error handling in Airtable run query (#2234) * Refactor error handling to streamline QueryError messages * Handle 404 errors * Enhance: Add AI-tag to datasources in marketplace page (#2597) * Add AI-tag to datasources in marketplace page * AI BANNER Tag * margin * Refactor tag rendering logic for marketplace page * Refactor tag rendering for datasource page * Refactor import and props * Remove ai prefix and fix indentations * Make custom hook for fetching plugins.json * Add AI tag on installed page * Marketplace page UI fix and add a plugin button fixes * Add AI tag on datasource connection form Change classname * Fix svg flickering * Fix svg chipping issue * Push AI tag to extreme right --------- Co-authored-by: Rudra <rudra21ultra@gmail.com> * Fix: Api call for token generation in client-credentials grant type (#2785) * fix: query kind for select source in rest api * Fix: ToolJet database limit check API issue (#11416) * bump to v3.0.5-ce * Added data-cy for newly added components (#11435) * Modified failed Platform cypress test cases for Tooljet V3 (#11486) * Modify platform cypress test cases * Added cypress test cases for user onboarding flow (#11499) * Add data-cy for newly added components * Add data-cy for onboarding page elements * Modify failed test cases * Adding more cases * Modify onboarding test cases * Modify user invite flow * chnages on onboarding test scripts * revert the changes * revert the changes * removed .only form profile file * resolved review changes --------- Co-authored-by: ajith-k-v <ajith.jaban@gmail.com> Co-authored-by: Sri mani Teja s <mani@Sris-MacBook-Pro-4.local> * fix: Fixes broken loading state for container * update cypress workflow for subpath * Add data-cy for workspace constants components (#11530) * fix table down load event not showing up * Hotfix: The build failed to include the reference file for custom validation of the ToolJet database schema. (#11490) * tooljet database schema custom validation reference file was not included in the build * fix: missed a dependency for copyfiles * change version to 3.1.0 --------- Co-authored-by: Ganesh Kumar <ganesh8056234@gmail.com> Co-authored-by: Parth <108089718+parthy007@users.noreply.github.com> Co-authored-by: Devanshu Rastogi <devanshu.rastogi05@gmail.com> Co-authored-by: Akshay Sasidharan <akshaysasidharan93@gmail.com> Co-authored-by: Mansukh Kaur <mansukhkaur@Mansukhs-MacBook-Pro.local> Co-authored-by: Rudra <rudra21ultra@gmail.com> Co-authored-by: Vijaykant Yadav <vjy239@gmail.com> Co-authored-by: Ganesh Kumar <40178541+ganesh8056@users.noreply.github.com> Co-authored-by: Ajith KV <ajith.jaban@gmail.com> Co-authored-by: Adish M <44204658+adishM98@users.noreply.github.com> Co-authored-by: Srimanitejas123 <mani@tooljet.com> Co-authored-by: Sri mani Teja s <mani@Sris-MacBook-Pro-4.local> Co-authored-by: Nithin David Thomas <1277421+nithindavid@users.noreply.github.com> Co-authored-by: Kartik Gupta <gupta.kartik18kg@gmail.com>
225 lines
7.6 KiB
TypeScript
225 lines
7.6 KiB
TypeScript
import { QueryResult, QueryService, QueryError, ConnectionTestResult } from '@tooljet-plugins/common';
|
|
const { MongoClient } = require('mongodb');
|
|
const JSON5 = require('json5');
|
|
import { EJSON } from 'bson';
|
|
import { SourceOptions, QueryOptions } from './types';
|
|
import tls from 'tls';
|
|
|
|
export default class MongodbService implements QueryService {
|
|
async run(sourceOptions: SourceOptions, queryOptions: QueryOptions, dataSourceId: string): Promise<QueryResult> {
|
|
const { db, close } = await this.getConnection(sourceOptions);
|
|
let result = {};
|
|
const operation = queryOptions.operation;
|
|
|
|
try {
|
|
switch (operation) {
|
|
case 'list_collections':
|
|
result = await db.listCollections().toArray();
|
|
break;
|
|
case 'insert_one':
|
|
result = await db
|
|
.collection(queryOptions.collection)
|
|
.insertOne(this.parseEJSON(queryOptions.document), this.parseEJSON(queryOptions.options));
|
|
break;
|
|
case 'insert_many':
|
|
result = await db
|
|
.collection(queryOptions.collection)
|
|
.insertMany(this.parseEJSON(queryOptions.documents), this.parseEJSON(queryOptions.options));
|
|
break;
|
|
case 'find_one':
|
|
result = await db
|
|
.collection(queryOptions.collection)
|
|
.findOne(this.parseEJSON(queryOptions.filter), this.parseEJSON(queryOptions.options));
|
|
break;
|
|
case 'find_many':
|
|
result = await db
|
|
.collection(queryOptions.collection)
|
|
.find(this.parseEJSON(queryOptions.filter), this.parseEJSON(queryOptions.options))
|
|
.toArray();
|
|
break;
|
|
case 'count_total':
|
|
result = await db
|
|
.collection(queryOptions.collection)
|
|
.estimatedDocumentCount(this.parseEJSON(queryOptions.options));
|
|
result = { count: result };
|
|
break;
|
|
case 'count':
|
|
result = await db
|
|
.collection(queryOptions.collection)
|
|
.countDocuments(this.parseEJSON(queryOptions.filter), this.parseEJSON(queryOptions.options));
|
|
result = { count: result };
|
|
break;
|
|
case 'distinct':
|
|
result = await db
|
|
.collection(queryOptions.collection)
|
|
.distinct(queryOptions.field, this.parseEJSON(queryOptions.filter), this.parseEJSON(queryOptions.options));
|
|
break;
|
|
case 'update_one':
|
|
result = await db
|
|
.collection(queryOptions.collection)
|
|
.updateOne(
|
|
this.parseEJSON(queryOptions.filter),
|
|
this.parseEJSON(queryOptions.update),
|
|
this.parseEJSON(queryOptions.options)
|
|
);
|
|
break;
|
|
case 'update_many':
|
|
result = await db
|
|
.collection(queryOptions.collection)
|
|
.updateMany(
|
|
this.parseEJSON(queryOptions.filter),
|
|
this.parseEJSON(queryOptions.update),
|
|
this.parseEJSON(queryOptions.options)
|
|
);
|
|
break;
|
|
case 'replace_one':
|
|
result = await db
|
|
.collection(queryOptions.collection)
|
|
.replaceOne(
|
|
this.parseEJSON(queryOptions.filter),
|
|
this.parseEJSON(queryOptions.replacement),
|
|
this.parseEJSON(queryOptions.options)
|
|
);
|
|
break;
|
|
case 'find_one_replace':
|
|
result = await db
|
|
.collection(queryOptions.collection)
|
|
.findOneAndReplace(
|
|
this.parseEJSON(queryOptions.filter),
|
|
this.parseEJSON(queryOptions.replacement),
|
|
this.parseEJSON(queryOptions.options)
|
|
);
|
|
break;
|
|
case 'find_one_update':
|
|
result = await db
|
|
.collection(queryOptions.collection)
|
|
.findOneAndUpdate(
|
|
this.parseEJSON(queryOptions.filter),
|
|
this.parseEJSON(queryOptions.update),
|
|
this.parseEJSON(queryOptions.options)
|
|
);
|
|
break;
|
|
case 'find_one_delete':
|
|
result = await db
|
|
.collection(queryOptions.collection)
|
|
.findOneAndDelete(this.parseEJSON(queryOptions.filter), this.parseEJSON(queryOptions.options));
|
|
break;
|
|
case 'delete_one':
|
|
result = await db
|
|
.collection(queryOptions.collection)
|
|
.deleteOne(this.parseEJSON(queryOptions.filter), this.parseEJSON(queryOptions.options));
|
|
break;
|
|
case 'delete_many':
|
|
result = await db
|
|
.collection(queryOptions.collection)
|
|
.deleteMany(this.parseEJSON(queryOptions.filter), this.parseEJSON(queryOptions.options));
|
|
break;
|
|
case 'bulk_write':
|
|
result = await db
|
|
.collection(queryOptions.collection)
|
|
.bulkWrite(this.parseEJSON(queryOptions.operations), this.parseEJSON(queryOptions.options));
|
|
break;
|
|
case 'aggregate':
|
|
result = await db
|
|
.collection(queryOptions.collection)
|
|
.aggregate(this.parseEJSON(queryOptions.pipeline), this.parseEJSON(queryOptions.options))
|
|
.toArray();
|
|
break;
|
|
}
|
|
} catch (err) {
|
|
console.log(err);
|
|
throw new QueryError('Query could not be completed', err.message, {});
|
|
} finally {
|
|
await close();
|
|
}
|
|
|
|
return {
|
|
status: 'ok',
|
|
data: result,
|
|
};
|
|
}
|
|
|
|
parseEJSON(maybeEJSON?: string): any {
|
|
if (!maybeEJSON) return {};
|
|
|
|
return EJSON.parse(JSON.stringify(JSON5.parse(maybeEJSON)));
|
|
}
|
|
|
|
async testConnection(sourceOptions: SourceOptions): Promise<ConnectionTestResult> {
|
|
const { db, close } = await this.getConnection(sourceOptions);
|
|
await db.listCollections().toArray();
|
|
await close();
|
|
|
|
return {
|
|
status: 'ok',
|
|
};
|
|
}
|
|
|
|
async getConnection(sourceOptions: SourceOptions): Promise<any> {
|
|
let db = null,
|
|
client;
|
|
const connectionType = sourceOptions['connection_type'];
|
|
|
|
if (connectionType === 'manual') {
|
|
const database = sourceOptions.database;
|
|
const host = sourceOptions.host;
|
|
const port = sourceOptions.port;
|
|
const username = encodeURIComponent(sourceOptions.username);
|
|
const password = encodeURIComponent(sourceOptions.password);
|
|
|
|
const needsAuthentication = username !== '' && password !== '';
|
|
const uri = needsAuthentication
|
|
? `mongodb://${username}:${password}@${host}:${port}`
|
|
: `mongodb://${host}:${port}`;
|
|
|
|
let clientOptions = {};
|
|
|
|
if (sourceOptions.tls_certificate === 'client_certificate') {
|
|
const secureContext = tls.createSecureContext({
|
|
ca: sourceOptions.ca_cert,
|
|
cert: sourceOptions.client_cert,
|
|
key: sourceOptions.client_key,
|
|
});
|
|
clientOptions = {
|
|
tls: true,
|
|
secureContext,
|
|
};
|
|
}
|
|
|
|
if (sourceOptions.tls_certificate === 'ca_certificate') {
|
|
const secureContext = tls.createSecureContext({
|
|
ca: sourceOptions.ca_cert,
|
|
});
|
|
|
|
clientOptions = {
|
|
tls: true,
|
|
secureContext,
|
|
};
|
|
}
|
|
|
|
client = new MongoClient(uri, clientOptions);
|
|
await client.connect();
|
|
|
|
db = client.db(database);
|
|
} else {
|
|
const connectionString = sourceOptions['connection_string'];
|
|
|
|
const password = connectionString.match(/(?<=:\/\/)(.*):(.*)@/)[2];
|
|
|
|
const encodedPassword = encodeURIComponent(password);
|
|
|
|
const encodedConnectionString = connectionString.replace(password, encodedPassword);
|
|
|
|
client = new MongoClient(encodedConnectionString, { useNewUrlParser: true, useUnifiedTopology: true });
|
|
await client.connect();
|
|
db = client.db();
|
|
}
|
|
|
|
return {
|
|
db,
|
|
close: async () => {
|
|
await client?.close?.();
|
|
},
|
|
};
|
|
}
|
|
}
|