ToolJet/plugins/packages/gcs/lib/operations.ts

67 lines
2.5 KiB
TypeScript
Raw Normal View History

Move plugins to root (#1728) * feat: move plugins to root * modify tsconfig * add .gitignore * delete old plugins file * add parcel * docker compose volume mount * add gcs * add typescript to plugins folder * gcs to ts * add dynamodb * add elastic search * add firestore * add gsheets * add graphql * add mongodb * mssql * add mysql * add postgresql * add redis * add s3 * add slack * add stripe * remove plugin related packages from pkgjson * add lib folder * add gitignore * remove typescript generated files * remove generated file * remove generated files * add twilio * add dist to docker compose cache binding * add dist prefix * cleanup - 1 * delete dist * rename to index.ts + add jest config * add it.todo in tests * test fixes * test file changes * fix type checks * add @tooljet/plugins to server package json * esm vs commonjs bug, reduce got to 11.8.2 from 12.0.0 * docker file npm package version fix * add typesense * cleaup - 2 * add sendgrid * add lerna build and clean script for all packages + tsconfig * cleanup -3 * add plugins build step * add missing plugins build step in npm run build * add mssql, mysql & postgres as singleton classes * add db connection to cache only if datasourceId is available * client: add data source schema/manifest files * add query operations files * logic for wrapping form with schema * add script to create index file * add @tooljet/plugins to frontend folder * cleanup 1 -frontend * cleanup - 2 // frontend // data queries * add client and index to gitignore * update gitignore * fix lint & test * update ci * fix unit, e2e * cleanup -3 * fix test * fix tests * fix indent * try npm ci * fix tests * fix typo * fix * rename file for server entry * heroku fix * add main and types entry points in pkg json * move common to root * cleanup - 4: remove redundant $ sign prefix * cleanup - 4: remove redundant $ sign prefix * update options in-sync before DOM is painted * change type cloud to cloud storage * update readme * update ci.yml * update ci yml * add pkg-lock.json * rename index.ts to server.ts * update lock files * add server package.lock * remove unused import * revert commit: add minio * add root dep * import server.ts * remove plugins build step * add npm shrinkwrap * update version - plugins * add new version - 0.0.8 * upgrade version * move to symlinked package * add lock file * feat: add icon inside package * add plugin creation docs * Remove seed * move icons to plugins folder * install pg dep * add react to packages * add seed cmd * revert change * add plugins build in lint, e2e, unit * e2e, lint use npm ci * update dockerfile for plugins * try combining release with web * limit memory on release * try executing seed script post transpile * try executing seed from server directory * update seed execution * add minio * add correct type * add minio to pkg json * remove old file * fix provider key * add python installable + npm ^7.2.0 (#1752) * add python installable + npm ^7.2.0 * add py to prod file * pin npm version to 7.20.0 * pin npm version to 7.20.0 * split into multi stage build and remove python for buildx * copy plugins from buider stage * update dependencies * add freetds dependency * update server dockerfile * update client dockerfile * update dev dockerfile and compose file * fix entrypoint * fix server dev dockerfile * update docker-compose * remove npm install on root dir on docker build * fix heroku script * make lerna prod dependency to enable prod builds * remove redundant env setup Co-authored-by: Akshay Sasidharan <akshaysasidharan93@gmail.com> Co-authored-by: navaneeth <navaneethpk@outlook.com>
2022-01-17 07:08:17 +00:00
import * as stream from 'stream';
export async function listBuckets(client, _options) {
const [buckets, ,] = await client.getBuckets({
autoPaginate: false,
});
return { buckets: buckets.map((bucket) => bucket.name) };
Move plugins to root (#1728) * feat: move plugins to root * modify tsconfig * add .gitignore * delete old plugins file * add parcel * docker compose volume mount * add gcs * add typescript to plugins folder * gcs to ts * add dynamodb * add elastic search * add firestore * add gsheets * add graphql * add mongodb * mssql * add mysql * add postgresql * add redis * add s3 * add slack * add stripe * remove plugin related packages from pkgjson * add lib folder * add gitignore * remove typescript generated files * remove generated file * remove generated files * add twilio * add dist to docker compose cache binding * add dist prefix * cleanup - 1 * delete dist * rename to index.ts + add jest config * add it.todo in tests * test fixes * test file changes * fix type checks * add @tooljet/plugins to server package json * esm vs commonjs bug, reduce got to 11.8.2 from 12.0.0 * docker file npm package version fix * add typesense * cleaup - 2 * add sendgrid * add lerna build and clean script for all packages + tsconfig * cleanup -3 * add plugins build step * add missing plugins build step in npm run build * add mssql, mysql & postgres as singleton classes * add db connection to cache only if datasourceId is available * client: add data source schema/manifest files * add query operations files * logic for wrapping form with schema * add script to create index file * add @tooljet/plugins to frontend folder * cleanup 1 -frontend * cleanup - 2 // frontend // data queries * add client and index to gitignore * update gitignore * fix lint & test * update ci * fix unit, e2e * cleanup -3 * fix test * fix tests * fix indent * try npm ci * fix tests * fix typo * fix * rename file for server entry * heroku fix * add main and types entry points in pkg json * move common to root * cleanup - 4: remove redundant $ sign prefix * cleanup - 4: remove redundant $ sign prefix * update options in-sync before DOM is painted * change type cloud to cloud storage * update readme * update ci.yml * update ci yml * add pkg-lock.json * rename index.ts to server.ts * update lock files * add server package.lock * remove unused import * revert commit: add minio * add root dep * import server.ts * remove plugins build step * add npm shrinkwrap * update version - plugins * add new version - 0.0.8 * upgrade version * move to symlinked package * add lock file * feat: add icon inside package * add plugin creation docs * Remove seed * move icons to plugins folder * install pg dep * add react to packages * add seed cmd * revert change * add plugins build in lint, e2e, unit * e2e, lint use npm ci * update dockerfile for plugins * try combining release with web * limit memory on release * try executing seed script post transpile * try executing seed from server directory * update seed execution * add minio * add correct type * add minio to pkg json * remove old file * fix provider key * add python installable + npm ^7.2.0 (#1752) * add python installable + npm ^7.2.0 * add py to prod file * pin npm version to 7.20.0 * pin npm version to 7.20.0 * split into multi stage build and remove python for buildx * copy plugins from buider stage * update dependencies * add freetds dependency * update server dockerfile * update client dockerfile * update dev dockerfile and compose file * fix entrypoint * fix server dev dockerfile * update docker-compose * remove npm install on root dir on docker build * fix heroku script * make lerna prod dependency to enable prod builds * remove redundant env setup Co-authored-by: Akshay Sasidharan <akshaysasidharan93@gmail.com> Co-authored-by: navaneeth <navaneethpk@outlook.com>
2022-01-17 07:08:17 +00:00
}
export async function listFiles(client, options) {
const [, , metadata] = await client
.bucket(options['bucket'])
.getFiles({ prefix: options['prefix'], autoPaginate: false });
return { files: metadata.items };
Move plugins to root (#1728) * feat: move plugins to root * modify tsconfig * add .gitignore * delete old plugins file * add parcel * docker compose volume mount * add gcs * add typescript to plugins folder * gcs to ts * add dynamodb * add elastic search * add firestore * add gsheets * add graphql * add mongodb * mssql * add mysql * add postgresql * add redis * add s3 * add slack * add stripe * remove plugin related packages from pkgjson * add lib folder * add gitignore * remove typescript generated files * remove generated file * remove generated files * add twilio * add dist to docker compose cache binding * add dist prefix * cleanup - 1 * delete dist * rename to index.ts + add jest config * add it.todo in tests * test fixes * test file changes * fix type checks * add @tooljet/plugins to server package json * esm vs commonjs bug, reduce got to 11.8.2 from 12.0.0 * docker file npm package version fix * add typesense * cleaup - 2 * add sendgrid * add lerna build and clean script for all packages + tsconfig * cleanup -3 * add plugins build step * add missing plugins build step in npm run build * add mssql, mysql & postgres as singleton classes * add db connection to cache only if datasourceId is available * client: add data source schema/manifest files * add query operations files * logic for wrapping form with schema * add script to create index file * add @tooljet/plugins to frontend folder * cleanup 1 -frontend * cleanup - 2 // frontend // data queries * add client and index to gitignore * update gitignore * fix lint & test * update ci * fix unit, e2e * cleanup -3 * fix test * fix tests * fix indent * try npm ci * fix tests * fix typo * fix * rename file for server entry * heroku fix * add main and types entry points in pkg json * move common to root * cleanup - 4: remove redundant $ sign prefix * cleanup - 4: remove redundant $ sign prefix * update options in-sync before DOM is painted * change type cloud to cloud storage * update readme * update ci.yml * update ci yml * add pkg-lock.json * rename index.ts to server.ts * update lock files * add server package.lock * remove unused import * revert commit: add minio * add root dep * import server.ts * remove plugins build step * add npm shrinkwrap * update version - plugins * add new version - 0.0.8 * upgrade version * move to symlinked package * add lock file * feat: add icon inside package * add plugin creation docs * Remove seed * move icons to plugins folder * install pg dep * add react to packages * add seed cmd * revert change * add plugins build in lint, e2e, unit * e2e, lint use npm ci * update dockerfile for plugins * try combining release with web * limit memory on release * try executing seed script post transpile * try executing seed from server directory * update seed execution * add minio * add correct type * add minio to pkg json * remove old file * fix provider key * add python installable + npm ^7.2.0 (#1752) * add python installable + npm ^7.2.0 * add py to prod file * pin npm version to 7.20.0 * pin npm version to 7.20.0 * split into multi stage build and remove python for buildx * copy plugins from buider stage * update dependencies * add freetds dependency * update server dockerfile * update client dockerfile * update dev dockerfile and compose file * fix entrypoint * fix server dev dockerfile * update docker-compose * remove npm install on root dir on docker build * fix heroku script * make lerna prod dependency to enable prod builds * remove redundant env setup Co-authored-by: Akshay Sasidharan <akshaysasidharan93@gmail.com> Co-authored-by: navaneeth <navaneethpk@outlook.com>
2022-01-17 07:08:17 +00:00
}
export async function getFile(client, options) {
// Create a helper function to convert a ReadableStream to a string.
const streamToString = (stream) =>
new Promise((resolve, reject) => {
const chunks = [];
stream.on('data', (chunk) => chunks.push(chunk));
stream.on('error', reject);
stream.on('end', () => resolve(Buffer.concat(chunks).toString('utf-8')));
Move plugins to root (#1728) * feat: move plugins to root * modify tsconfig * add .gitignore * delete old plugins file * add parcel * docker compose volume mount * add gcs * add typescript to plugins folder * gcs to ts * add dynamodb * add elastic search * add firestore * add gsheets * add graphql * add mongodb * mssql * add mysql * add postgresql * add redis * add s3 * add slack * add stripe * remove plugin related packages from pkgjson * add lib folder * add gitignore * remove typescript generated files * remove generated file * remove generated files * add twilio * add dist to docker compose cache binding * add dist prefix * cleanup - 1 * delete dist * rename to index.ts + add jest config * add it.todo in tests * test fixes * test file changes * fix type checks * add @tooljet/plugins to server package json * esm vs commonjs bug, reduce got to 11.8.2 from 12.0.0 * docker file npm package version fix * add typesense * cleaup - 2 * add sendgrid * add lerna build and clean script for all packages + tsconfig * cleanup -3 * add plugins build step * add missing plugins build step in npm run build * add mssql, mysql & postgres as singleton classes * add db connection to cache only if datasourceId is available * client: add data source schema/manifest files * add query operations files * logic for wrapping form with schema * add script to create index file * add @tooljet/plugins to frontend folder * cleanup 1 -frontend * cleanup - 2 // frontend // data queries * add client and index to gitignore * update gitignore * fix lint & test * update ci * fix unit, e2e * cleanup -3 * fix test * fix tests * fix indent * try npm ci * fix tests * fix typo * fix * rename file for server entry * heroku fix * add main and types entry points in pkg json * move common to root * cleanup - 4: remove redundant $ sign prefix * cleanup - 4: remove redundant $ sign prefix * update options in-sync before DOM is painted * change type cloud to cloud storage * update readme * update ci.yml * update ci yml * add pkg-lock.json * rename index.ts to server.ts * update lock files * add server package.lock * remove unused import * revert commit: add minio * add root dep * import server.ts * remove plugins build step * add npm shrinkwrap * update version - plugins * add new version - 0.0.8 * upgrade version * move to symlinked package * add lock file * feat: add icon inside package * add plugin creation docs * Remove seed * move icons to plugins folder * install pg dep * add react to packages * add seed cmd * revert change * add plugins build in lint, e2e, unit * e2e, lint use npm ci * update dockerfile for plugins * try combining release with web * limit memory on release * try executing seed script post transpile * try executing seed from server directory * update seed execution * add minio * add correct type * add minio to pkg json * remove old file * fix provider key * add python installable + npm ^7.2.0 (#1752) * add python installable + npm ^7.2.0 * add py to prod file * pin npm version to 7.20.0 * pin npm version to 7.20.0 * split into multi stage build and remove python for buildx * copy plugins from buider stage * update dependencies * add freetds dependency * update server dockerfile * update client dockerfile * update dev dockerfile and compose file * fix entrypoint * fix server dev dockerfile * update docker-compose * remove npm install on root dir on docker build * fix heroku script * make lerna prod dependency to enable prod builds * remove redundant env setup Co-authored-by: Akshay Sasidharan <akshaysasidharan93@gmail.com> Co-authored-by: navaneeth <navaneethpk@outlook.com>
2022-01-17 07:08:17 +00:00
});
const data = client.bucket(options['bucket']).file(options['file']).createReadStream();
// Convert the ReadableStream to a string.
const bodyContents = await streamToString(data);
return Object.assign(Object.assign({}, data), { Body: bodyContents });
Move plugins to root (#1728) * feat: move plugins to root * modify tsconfig * add .gitignore * delete old plugins file * add parcel * docker compose volume mount * add gcs * add typescript to plugins folder * gcs to ts * add dynamodb * add elastic search * add firestore * add gsheets * add graphql * add mongodb * mssql * add mysql * add postgresql * add redis * add s3 * add slack * add stripe * remove plugin related packages from pkgjson * add lib folder * add gitignore * remove typescript generated files * remove generated file * remove generated files * add twilio * add dist to docker compose cache binding * add dist prefix * cleanup - 1 * delete dist * rename to index.ts + add jest config * add it.todo in tests * test fixes * test file changes * fix type checks * add @tooljet/plugins to server package json * esm vs commonjs bug, reduce got to 11.8.2 from 12.0.0 * docker file npm package version fix * add typesense * cleaup - 2 * add sendgrid * add lerna build and clean script for all packages + tsconfig * cleanup -3 * add plugins build step * add missing plugins build step in npm run build * add mssql, mysql & postgres as singleton classes * add db connection to cache only if datasourceId is available * client: add data source schema/manifest files * add query operations files * logic for wrapping form with schema * add script to create index file * add @tooljet/plugins to frontend folder * cleanup 1 -frontend * cleanup - 2 // frontend // data queries * add client and index to gitignore * update gitignore * fix lint & test * update ci * fix unit, e2e * cleanup -3 * fix test * fix tests * fix indent * try npm ci * fix tests * fix typo * fix * rename file for server entry * heroku fix * add main and types entry points in pkg json * move common to root * cleanup - 4: remove redundant $ sign prefix * cleanup - 4: remove redundant $ sign prefix * update options in-sync before DOM is painted * change type cloud to cloud storage * update readme * update ci.yml * update ci yml * add pkg-lock.json * rename index.ts to server.ts * update lock files * add server package.lock * remove unused import * revert commit: add minio * add root dep * import server.ts * remove plugins build step * add npm shrinkwrap * update version - plugins * add new version - 0.0.8 * upgrade version * move to symlinked package * add lock file * feat: add icon inside package * add plugin creation docs * Remove seed * move icons to plugins folder * install pg dep * add react to packages * add seed cmd * revert change * add plugins build in lint, e2e, unit * e2e, lint use npm ci * update dockerfile for plugins * try combining release with web * limit memory on release * try executing seed script post transpile * try executing seed from server directory * update seed execution * add minio * add correct type * add minio to pkg json * remove old file * fix provider key * add python installable + npm ^7.2.0 (#1752) * add python installable + npm ^7.2.0 * add py to prod file * pin npm version to 7.20.0 * pin npm version to 7.20.0 * split into multi stage build and remove python for buildx * copy plugins from buider stage * update dependencies * add freetds dependency * update server dockerfile * update client dockerfile * update dev dockerfile and compose file * fix entrypoint * fix server dev dockerfile * update docker-compose * remove npm install on root dir on docker build * fix heroku script * make lerna prod dependency to enable prod builds * remove redundant env setup Co-authored-by: Akshay Sasidharan <akshaysasidharan93@gmail.com> Co-authored-by: navaneeth <navaneethpk@outlook.com>
2022-01-17 07:08:17 +00:00
}
export async function uploadFile(client, options) {
// Get a reference to the bucket
const myBucket = client.bucket(options['bucket']);
// Create a reference to a file object
const file = myBucket.file(options['file']);
// Create a pass through stream from a string
const passthroughStream = new stream.PassThrough();
passthroughStream.end(Buffer.from(options['data'], options['encoding'] || 'utf8'));
passthroughStream.pipe(
file.createWriteStream({
metadata: { contentType: options['contentType'] },
})
);
return { success: true };
Move plugins to root (#1728) * feat: move plugins to root * modify tsconfig * add .gitignore * delete old plugins file * add parcel * docker compose volume mount * add gcs * add typescript to plugins folder * gcs to ts * add dynamodb * add elastic search * add firestore * add gsheets * add graphql * add mongodb * mssql * add mysql * add postgresql * add redis * add s3 * add slack * add stripe * remove plugin related packages from pkgjson * add lib folder * add gitignore * remove typescript generated files * remove generated file * remove generated files * add twilio * add dist to docker compose cache binding * add dist prefix * cleanup - 1 * delete dist * rename to index.ts + add jest config * add it.todo in tests * test fixes * test file changes * fix type checks * add @tooljet/plugins to server package json * esm vs commonjs bug, reduce got to 11.8.2 from 12.0.0 * docker file npm package version fix * add typesense * cleaup - 2 * add sendgrid * add lerna build and clean script for all packages + tsconfig * cleanup -3 * add plugins build step * add missing plugins build step in npm run build * add mssql, mysql & postgres as singleton classes * add db connection to cache only if datasourceId is available * client: add data source schema/manifest files * add query operations files * logic for wrapping form with schema * add script to create index file * add @tooljet/plugins to frontend folder * cleanup 1 -frontend * cleanup - 2 // frontend // data queries * add client and index to gitignore * update gitignore * fix lint & test * update ci * fix unit, e2e * cleanup -3 * fix test * fix tests * fix indent * try npm ci * fix tests * fix typo * fix * rename file for server entry * heroku fix * add main and types entry points in pkg json * move common to root * cleanup - 4: remove redundant $ sign prefix * cleanup - 4: remove redundant $ sign prefix * update options in-sync before DOM is painted * change type cloud to cloud storage * update readme * update ci.yml * update ci yml * add pkg-lock.json * rename index.ts to server.ts * update lock files * add server package.lock * remove unused import * revert commit: add minio * add root dep * import server.ts * remove plugins build step * add npm shrinkwrap * update version - plugins * add new version - 0.0.8 * upgrade version * move to symlinked package * add lock file * feat: add icon inside package * add plugin creation docs * Remove seed * move icons to plugins folder * install pg dep * add react to packages * add seed cmd * revert change * add plugins build in lint, e2e, unit * e2e, lint use npm ci * update dockerfile for plugins * try combining release with web * limit memory on release * try executing seed script post transpile * try executing seed from server directory * update seed execution * add minio * add correct type * add minio to pkg json * remove old file * fix provider key * add python installable + npm ^7.2.0 (#1752) * add python installable + npm ^7.2.0 * add py to prod file * pin npm version to 7.20.0 * pin npm version to 7.20.0 * split into multi stage build and remove python for buildx * copy plugins from buider stage * update dependencies * add freetds dependency * update server dockerfile * update client dockerfile * update dev dockerfile and compose file * fix entrypoint * fix server dev dockerfile * update docker-compose * remove npm install on root dir on docker build * fix heroku script * make lerna prod dependency to enable prod builds * remove redundant env setup Co-authored-by: Akshay Sasidharan <akshaysasidharan93@gmail.com> Co-authored-by: navaneeth <navaneethpk@outlook.com>
2022-01-17 07:08:17 +00:00
}
export async function signedUrlForGet(client, options) {
const defaultExpiry = Date.now() + 15 * 60 * 1000; // 15 mins
const expiresIn = options['expiresIn'] ? Date.now() + options['expiresIn'] * 1000 : defaultExpiry;
const [url] = await client.bucket(options['bucket']).file(options['file']).getSignedUrl({
version: 'v4',
action: 'read',
expires: expiresIn,
});
return { url };
Move plugins to root (#1728) * feat: move plugins to root * modify tsconfig * add .gitignore * delete old plugins file * add parcel * docker compose volume mount * add gcs * add typescript to plugins folder * gcs to ts * add dynamodb * add elastic search * add firestore * add gsheets * add graphql * add mongodb * mssql * add mysql * add postgresql * add redis * add s3 * add slack * add stripe * remove plugin related packages from pkgjson * add lib folder * add gitignore * remove typescript generated files * remove generated file * remove generated files * add twilio * add dist to docker compose cache binding * add dist prefix * cleanup - 1 * delete dist * rename to index.ts + add jest config * add it.todo in tests * test fixes * test file changes * fix type checks * add @tooljet/plugins to server package json * esm vs commonjs bug, reduce got to 11.8.2 from 12.0.0 * docker file npm package version fix * add typesense * cleaup - 2 * add sendgrid * add lerna build and clean script for all packages + tsconfig * cleanup -3 * add plugins build step * add missing plugins build step in npm run build * add mssql, mysql & postgres as singleton classes * add db connection to cache only if datasourceId is available * client: add data source schema/manifest files * add query operations files * logic for wrapping form with schema * add script to create index file * add @tooljet/plugins to frontend folder * cleanup 1 -frontend * cleanup - 2 // frontend // data queries * add client and index to gitignore * update gitignore * fix lint & test * update ci * fix unit, e2e * cleanup -3 * fix test * fix tests * fix indent * try npm ci * fix tests * fix typo * fix * rename file for server entry * heroku fix * add main and types entry points in pkg json * move common to root * cleanup - 4: remove redundant $ sign prefix * cleanup - 4: remove redundant $ sign prefix * update options in-sync before DOM is painted * change type cloud to cloud storage * update readme * update ci.yml * update ci yml * add pkg-lock.json * rename index.ts to server.ts * update lock files * add server package.lock * remove unused import * revert commit: add minio * add root dep * import server.ts * remove plugins build step * add npm shrinkwrap * update version - plugins * add new version - 0.0.8 * upgrade version * move to symlinked package * add lock file * feat: add icon inside package * add plugin creation docs * Remove seed * move icons to plugins folder * install pg dep * add react to packages * add seed cmd * revert change * add plugins build in lint, e2e, unit * e2e, lint use npm ci * update dockerfile for plugins * try combining release with web * limit memory on release * try executing seed script post transpile * try executing seed from server directory * update seed execution * add minio * add correct type * add minio to pkg json * remove old file * fix provider key * add python installable + npm ^7.2.0 (#1752) * add python installable + npm ^7.2.0 * add py to prod file * pin npm version to 7.20.0 * pin npm version to 7.20.0 * split into multi stage build and remove python for buildx * copy plugins from buider stage * update dependencies * add freetds dependency * update server dockerfile * update client dockerfile * update dev dockerfile and compose file * fix entrypoint * fix server dev dockerfile * update docker-compose * remove npm install on root dir on docker build * fix heroku script * make lerna prod dependency to enable prod builds * remove redundant env setup Co-authored-by: Akshay Sasidharan <akshaysasidharan93@gmail.com> Co-authored-by: navaneeth <navaneethpk@outlook.com>
2022-01-17 07:08:17 +00:00
}
export async function signedUrlForPut(client, options) {
const defaultExpiry = Date.now() + 15 * 60 * 1000; // 15 mins
const expiresIn = options['expiresIn'] ? Date.now() + options['expiresIn'] * 1000 : defaultExpiry;
const [url] = await client
.bucket(options['bucket'])
.file(options['file'])
.getSignedUrl(
Object.assign(
{ version: 'v4', action: 'write', expires: expiresIn },
options['contentType'] && { contentType: options['contentType'] }
)
);
return { url };
}