ToolJet/server/data-migration-config.ts

55 lines
1.3 KiB
TypeScript
Raw Normal View History

import { TypeOrmModuleOptions } from '@nestjs/typeorm';
2022-12-23 12:58:10 +00:00
import { getEnvVars } from './scripts/database-config-utils';
Feature: Tooljet Database (#4951) * wip * internal db per workspace * fix async query * feat: add storage layer route * feat: add drawer component * feat: add react-table to load data * feat: add columns form * feat: add create column form, create row form * feat: add postgrest js * add tooljet db controller to proxy requests to postgrest * util: add postgrest filter builder helper utility * feat: add filter popover * use helper utility for building query * add sortable filters * add box shadow for filter popup * use overlay trigger * use react select * add new column addition * add dropdown for table header, table list * Move filter.jsx * feat: add sort popover * feat: add postgrest js .order fn * setup tooljetdb with restricted grants for users * make db schemas added loaded dynamically on postgrest server * fix query * sign jwt token to auth user at postgrest * update db schema user with workspace * chore: add table listing * update data and columns from api * feat: add context api for sharing data * add ability to create table, view tables and add columns * use columns for sort from context api * fix ormconfig * feat: add table listing integration * feat: add create table integration * fix for rds deployment * add internal table translation instead of schema * remove tooljetdb as a datasource * wrap placeholder on proxy query * add active workspace guard * scope tooljetdb by workspace * update active workspace guard * seperate proxy related concerns to different service * make use of org id param * rename storage layer to tooljet databse * update specs * feat: Update list when new table added * feat: add create column * chore: add orgId to url + misc changes * chore: move popover to separate file * remove unused var * rename files * feat: add multiple columns * feat: add new row * removes postgrest-js from pkg lock * feat: add row data * feat: add sorting * feat: allow row deletion * feat: add search * feat: add filtering * feat: add edit mode * feat: add columns while edit table * add view table action * update setup for column constraint * fix query * integrate view_table, primary key field * render toogle for boolean data type * update view table query for primary key * fetch metadata refactor * add capability to set default values * feat: allow deletion of record based on primary key * feat: add default value while creating column * send query from sort & filter component * css changes * allow empty data * add requested changes * add err message * add common fn * allow sort + filter * remove unwanted defaults key * css changes * add more operators * dark mode fixes * add drawer footer * add loader for list tables * add dashboard design changes * design changes * add capability to drop table and delete column * add breadcrumbs * design changes * add profile * refactor tooljetdb controller * update routes * add empty page changes * delete column fix * fix delete column * design changes * fetch tables post delete * homepage changes * hide ellipsis on hover * add org settings page * add edit + create org * add notification center * fix: group permissions switch issue * add logo * remove anchor tag * fix merge conflicts * css changes * add err boundary * setup query editor * css changes * fix: merge conflicts * add menuPortal prop to filterform and sort form * fix seed * fix: build * design changes * design changes + refactor code * fix imports * fix: drawer issue on delete table * add search box changes * fix: tablename max-length 255 * fix: set newly created table as selected item * remove edit column option * added badges to enterprise only features * disable edit column * table styles * fix: popover position, placeholder default * fix: display boolean values in table * fix: tooljet database default type values * css changes * add query manager for tooljet db with create and list row * dark mode fixes * remove Header component * add ability to delete tooljetdb rows from query manager * add ability to update tooljetdb rows * dark mode fixes * css changes * display actions icon on hover * folder onclick change * add empty page styles * fix proxy requests * feat: randomize icon creation * add max items per page prop for pagination * removes unwanted position attr * add table name validation + disable submit btn while api fetch * [Bugfix] internal storage toast | trigger toasts for running preview db queries (#5019) * resolves: no toasts are fired when preview query is run for db queries * fire success toast for created and no content status text for query success * remove invalid migration * skip migration if tooljet db already created * fix: app clone icon param * fix: show confirmation box if filter options are empty in query (#5021) * for now: show native confirmation box of the brower to confirm the delete all query * typo * Revert "typo" This reverts commit b5ce5ed8890056974395750b6e07475390748e3b. * cleaned * cleaned * show confirmation box if filter options are empty in query * [Refactor/Bugfix] database query (#5028) * refactored list rows operations * remove unwanted cls * refactor create row * reafactored update rows * refactored delete rows * padding fix for tj-query * add static templates * review changes * remove unused file * Chore: tooljetdb render setup (#5033) * add postgrest for render preview deploy * pin version * add healthCheckPath * remove health check * handle database url parsing db params * add defaults for tooljetdb env * fix hostname * handle env in migration files * refactor dbconfig build * fix pg db usage * add parsed env context * add tooljetdb env * refactor db config utils Co-authored-by: gandharv <gandharvkumargarg@gmail.com> Co-authored-by: Shubhendra <withshubh@gmail.com> Co-authored-by: Arpit <arpitnath42@gmail.com>
2022-12-22 20:39:57 +00:00
2023-01-03 07:55:45 +00:00
function sslConfig(envVars) {
let config = {};
if (envVars?.DATABASE_URL)
config = {
url: envVars.DATABASE_URL,
ssl: { rejectUnauthorized: false },
2023-01-03 07:55:45 +00:00
};
if (envVars?.CA_CERT)
config = {
...config,
...{ ssl: { rejectUnauthorized: false, ca: envVars.CA_CERT } },
};
return config;
}
Feature: Tooljet Database (#4951) * wip * internal db per workspace * fix async query * feat: add storage layer route * feat: add drawer component * feat: add react-table to load data * feat: add columns form * feat: add create column form, create row form * feat: add postgrest js * add tooljet db controller to proxy requests to postgrest * util: add postgrest filter builder helper utility * feat: add filter popover * use helper utility for building query * add sortable filters * add box shadow for filter popup * use overlay trigger * use react select * add new column addition * add dropdown for table header, table list * Move filter.jsx * feat: add sort popover * feat: add postgrest js .order fn * setup tooljetdb with restricted grants for users * make db schemas added loaded dynamically on postgrest server * fix query * sign jwt token to auth user at postgrest * update db schema user with workspace * chore: add table listing * update data and columns from api * feat: add context api for sharing data * add ability to create table, view tables and add columns * use columns for sort from context api * fix ormconfig * feat: add table listing integration * feat: add create table integration * fix for rds deployment * add internal table translation instead of schema * remove tooljetdb as a datasource * wrap placeholder on proxy query * add active workspace guard * scope tooljetdb by workspace * update active workspace guard * seperate proxy related concerns to different service * make use of org id param * rename storage layer to tooljet databse * update specs * feat: Update list when new table added * feat: add create column * chore: add orgId to url + misc changes * chore: move popover to separate file * remove unused var * rename files * feat: add multiple columns * feat: add new row * removes postgrest-js from pkg lock * feat: add row data * feat: add sorting * feat: allow row deletion * feat: add search * feat: add filtering * feat: add edit mode * feat: add columns while edit table * add view table action * update setup for column constraint * fix query * integrate view_table, primary key field * render toogle for boolean data type * update view table query for primary key * fetch metadata refactor * add capability to set default values * feat: allow deletion of record based on primary key * feat: add default value while creating column * send query from sort & filter component * css changes * allow empty data * add requested changes * add err message * add common fn * allow sort + filter * remove unwanted defaults key * css changes * add more operators * dark mode fixes * add drawer footer * add loader for list tables * add dashboard design changes * design changes * add capability to drop table and delete column * add breadcrumbs * design changes * add profile * refactor tooljetdb controller * update routes * add empty page changes * delete column fix * fix delete column * design changes * fetch tables post delete * homepage changes * hide ellipsis on hover * add org settings page * add edit + create org * add notification center * fix: group permissions switch issue * add logo * remove anchor tag * fix merge conflicts * css changes * add err boundary * setup query editor * css changes * fix: merge conflicts * add menuPortal prop to filterform and sort form * fix seed * fix: build * design changes * design changes + refactor code * fix imports * fix: drawer issue on delete table * add search box changes * fix: tablename max-length 255 * fix: set newly created table as selected item * remove edit column option * added badges to enterprise only features * disable edit column * table styles * fix: popover position, placeholder default * fix: display boolean values in table * fix: tooljet database default type values * css changes * add query manager for tooljet db with create and list row * dark mode fixes * remove Header component * add ability to delete tooljetdb rows from query manager * add ability to update tooljetdb rows * dark mode fixes * css changes * display actions icon on hover * folder onclick change * add empty page styles * fix proxy requests * feat: randomize icon creation * add max items per page prop for pagination * removes unwanted position attr * add table name validation + disable submit btn while api fetch * [Bugfix] internal storage toast | trigger toasts for running preview db queries (#5019) * resolves: no toasts are fired when preview query is run for db queries * fire success toast for created and no content status text for query success * remove invalid migration * skip migration if tooljet db already created * fix: app clone icon param * fix: show confirmation box if filter options are empty in query (#5021) * for now: show native confirmation box of the brower to confirm the delete all query * typo * Revert "typo" This reverts commit b5ce5ed8890056974395750b6e07475390748e3b. * cleaned * cleaned * show confirmation box if filter options are empty in query * [Refactor/Bugfix] database query (#5028) * refactored list rows operations * remove unwanted cls * refactor create row * reafactored update rows * refactored delete rows * padding fix for tj-query * add static templates * review changes * remove unused file * Chore: tooljetdb render setup (#5033) * add postgrest for render preview deploy * pin version * add healthCheckPath * remove health check * handle database url parsing db params * add defaults for tooljetdb env * fix hostname * handle env in migration files * refactor dbconfig build * fix pg db usage * add parsed env context * add tooljetdb env * refactor db config utils Co-authored-by: gandharv <gandharvkumargarg@gmail.com> Co-authored-by: Shubhendra <withshubh@gmail.com> Co-authored-by: Arpit <arpitnath42@gmail.com>
2022-12-22 20:39:57 +00:00
function buildConnectionOptions(): TypeOrmModuleOptions {
const data = getEnvVars();
const connectionParams = {
database: data.PG_DB,
port: +data.PG_PORT || 5432,
username: data.PG_USER,
password: data.PG_PASS,
host: data.PG_HOST,
connectTimeoutMS: 5000,
extra: {
max: 25,
},
2023-01-03 07:55:45 +00:00
...sslConfig(data),
Feature: Tooljet Database (#4951) * wip * internal db per workspace * fix async query * feat: add storage layer route * feat: add drawer component * feat: add react-table to load data * feat: add columns form * feat: add create column form, create row form * feat: add postgrest js * add tooljet db controller to proxy requests to postgrest * util: add postgrest filter builder helper utility * feat: add filter popover * use helper utility for building query * add sortable filters * add box shadow for filter popup * use overlay trigger * use react select * add new column addition * add dropdown for table header, table list * Move filter.jsx * feat: add sort popover * feat: add postgrest js .order fn * setup tooljetdb with restricted grants for users * make db schemas added loaded dynamically on postgrest server * fix query * sign jwt token to auth user at postgrest * update db schema user with workspace * chore: add table listing * update data and columns from api * feat: add context api for sharing data * add ability to create table, view tables and add columns * use columns for sort from context api * fix ormconfig * feat: add table listing integration * feat: add create table integration * fix for rds deployment * add internal table translation instead of schema * remove tooljetdb as a datasource * wrap placeholder on proxy query * add active workspace guard * scope tooljetdb by workspace * update active workspace guard * seperate proxy related concerns to different service * make use of org id param * rename storage layer to tooljet databse * update specs * feat: Update list when new table added * feat: add create column * chore: add orgId to url + misc changes * chore: move popover to separate file * remove unused var * rename files * feat: add multiple columns * feat: add new row * removes postgrest-js from pkg lock * feat: add row data * feat: add sorting * feat: allow row deletion * feat: add search * feat: add filtering * feat: add edit mode * feat: add columns while edit table * add view table action * update setup for column constraint * fix query * integrate view_table, primary key field * render toogle for boolean data type * update view table query for primary key * fetch metadata refactor * add capability to set default values * feat: allow deletion of record based on primary key * feat: add default value while creating column * send query from sort & filter component * css changes * allow empty data * add requested changes * add err message * add common fn * allow sort + filter * remove unwanted defaults key * css changes * add more operators * dark mode fixes * add drawer footer * add loader for list tables * add dashboard design changes * design changes * add capability to drop table and delete column * add breadcrumbs * design changes * add profile * refactor tooljetdb controller * update routes * add empty page changes * delete column fix * fix delete column * design changes * fetch tables post delete * homepage changes * hide ellipsis on hover * add org settings page * add edit + create org * add notification center * fix: group permissions switch issue * add logo * remove anchor tag * fix merge conflicts * css changes * add err boundary * setup query editor * css changes * fix: merge conflicts * add menuPortal prop to filterform and sort form * fix seed * fix: build * design changes * design changes + refactor code * fix imports * fix: drawer issue on delete table * add search box changes * fix: tablename max-length 255 * fix: set newly created table as selected item * remove edit column option * added badges to enterprise only features * disable edit column * table styles * fix: popover position, placeholder default * fix: display boolean values in table * fix: tooljet database default type values * css changes * add query manager for tooljet db with create and list row * dark mode fixes * remove Header component * add ability to delete tooljetdb rows from query manager * add ability to update tooljetdb rows * dark mode fixes * css changes * display actions icon on hover * folder onclick change * add empty page styles * fix proxy requests * feat: randomize icon creation * add max items per page prop for pagination * removes unwanted position attr * add table name validation + disable submit btn while api fetch * [Bugfix] internal storage toast | trigger toasts for running preview db queries (#5019) * resolves: no toasts are fired when preview query is run for db queries * fire success toast for created and no content status text for query success * remove invalid migration * skip migration if tooljet db already created * fix: app clone icon param * fix: show confirmation box if filter options are empty in query (#5021) * for now: show native confirmation box of the brower to confirm the delete all query * typo * Revert "typo" This reverts commit b5ce5ed8890056974395750b6e07475390748e3b. * cleaned * cleaned * show confirmation box if filter options are empty in query * [Refactor/Bugfix] database query (#5028) * refactored list rows operations * remove unwanted cls * refactor create row * reafactored update rows * refactored delete rows * padding fix for tj-query * add static templates * review changes * remove unused file * Chore: tooljetdb render setup (#5033) * add postgrest for render preview deploy * pin version * add healthCheckPath * remove health check * handle database url parsing db params * add defaults for tooljetdb env * fix hostname * handle env in migration files * refactor dbconfig build * fix pg db usage * add parsed env context * add tooljetdb env * refactor db config utils Co-authored-by: gandharv <gandharvkumargarg@gmail.com> Co-authored-by: Shubhendra <withshubh@gmail.com> Co-authored-by: Arpit <arpitnath42@gmail.com>
2022-12-22 20:39:57 +00:00
};
const entitiesDir =
data?.NODE_ENV === 'test' ? [__dirname + '/**/*.entity.ts'] : [__dirname + '/**/*.entity{.js,.ts}'];
return {
2022-12-23 12:58:10 +00:00
type: 'postgres',
...connectionParams,
entities: entitiesDir,
synchronize: false,
2022-12-23 12:58:10 +00:00
uuidExtension: 'pgcrypto',
migrationsRun: false,
2022-12-23 12:58:10 +00:00
migrationsTransactionMode: 'all',
logging: data.ORM_LOGGING || false,
2022-12-23 12:58:10 +00:00
migrations: [__dirname + '/data-migrations/**/*{.ts,.js}'],
};
}
Feature: Tooljet Database (#4951) * wip * internal db per workspace * fix async query * feat: add storage layer route * feat: add drawer component * feat: add react-table to load data * feat: add columns form * feat: add create column form, create row form * feat: add postgrest js * add tooljet db controller to proxy requests to postgrest * util: add postgrest filter builder helper utility * feat: add filter popover * use helper utility for building query * add sortable filters * add box shadow for filter popup * use overlay trigger * use react select * add new column addition * add dropdown for table header, table list * Move filter.jsx * feat: add sort popover * feat: add postgrest js .order fn * setup tooljetdb with restricted grants for users * make db schemas added loaded dynamically on postgrest server * fix query * sign jwt token to auth user at postgrest * update db schema user with workspace * chore: add table listing * update data and columns from api * feat: add context api for sharing data * add ability to create table, view tables and add columns * use columns for sort from context api * fix ormconfig * feat: add table listing integration * feat: add create table integration * fix for rds deployment * add internal table translation instead of schema * remove tooljetdb as a datasource * wrap placeholder on proxy query * add active workspace guard * scope tooljetdb by workspace * update active workspace guard * seperate proxy related concerns to different service * make use of org id param * rename storage layer to tooljet databse * update specs * feat: Update list when new table added * feat: add create column * chore: add orgId to url + misc changes * chore: move popover to separate file * remove unused var * rename files * feat: add multiple columns * feat: add new row * removes postgrest-js from pkg lock * feat: add row data * feat: add sorting * feat: allow row deletion * feat: add search * feat: add filtering * feat: add edit mode * feat: add columns while edit table * add view table action * update setup for column constraint * fix query * integrate view_table, primary key field * render toogle for boolean data type * update view table query for primary key * fetch metadata refactor * add capability to set default values * feat: allow deletion of record based on primary key * feat: add default value while creating column * send query from sort & filter component * css changes * allow empty data * add requested changes * add err message * add common fn * allow sort + filter * remove unwanted defaults key * css changes * add more operators * dark mode fixes * add drawer footer * add loader for list tables * add dashboard design changes * design changes * add capability to drop table and delete column * add breadcrumbs * design changes * add profile * refactor tooljetdb controller * update routes * add empty page changes * delete column fix * fix delete column * design changes * fetch tables post delete * homepage changes * hide ellipsis on hover * add org settings page * add edit + create org * add notification center * fix: group permissions switch issue * add logo * remove anchor tag * fix merge conflicts * css changes * add err boundary * setup query editor * css changes * fix: merge conflicts * add menuPortal prop to filterform and sort form * fix seed * fix: build * design changes * design changes + refactor code * fix imports * fix: drawer issue on delete table * add search box changes * fix: tablename max-length 255 * fix: set newly created table as selected item * remove edit column option * added badges to enterprise only features * disable edit column * table styles * fix: popover position, placeholder default * fix: display boolean values in table * fix: tooljet database default type values * css changes * add query manager for tooljet db with create and list row * dark mode fixes * remove Header component * add ability to delete tooljetdb rows from query manager * add ability to update tooljetdb rows * dark mode fixes * css changes * display actions icon on hover * folder onclick change * add empty page styles * fix proxy requests * feat: randomize icon creation * add max items per page prop for pagination * removes unwanted position attr * add table name validation + disable submit btn while api fetch * [Bugfix] internal storage toast | trigger toasts for running preview db queries (#5019) * resolves: no toasts are fired when preview query is run for db queries * fire success toast for created and no content status text for query success * remove invalid migration * skip migration if tooljet db already created * fix: app clone icon param * fix: show confirmation box if filter options are empty in query (#5021) * for now: show native confirmation box of the brower to confirm the delete all query * typo * Revert "typo" This reverts commit b5ce5ed8890056974395750b6e07475390748e3b. * cleaned * cleaned * show confirmation box if filter options are empty in query * [Refactor/Bugfix] database query (#5028) * refactored list rows operations * remove unwanted cls * refactor create row * reafactored update rows * refactored delete rows * padding fix for tj-query * add static templates * review changes * remove unused file * Chore: tooljetdb render setup (#5033) * add postgrest for render preview deploy * pin version * add healthCheckPath * remove health check * handle database url parsing db params * add defaults for tooljetdb env * fix hostname * handle env in migration files * refactor dbconfig build * fix pg db usage * add parsed env context * add tooljetdb env * refactor db config utils Co-authored-by: gandharv <gandharvkumargarg@gmail.com> Co-authored-by: Shubhendra <withshubh@gmail.com> Co-authored-by: Arpit <arpitnath42@gmail.com>
2022-12-22 20:39:57 +00:00
const ormconfig: TypeOrmModuleOptions = buildConnectionOptions();
export default ormconfig;