Commit graph

59 commits

Author SHA1 Message Date
Adish M
1e7329ab89 fix: update environment variables and improve setup scripts for production deployment 2026-02-19 11:38:38 +05:30
Adish M
3674a25f14 update default AWS region from us-east-1 to us-west-1 in Packer build configuration 2026-02-18 13:24:51 +05:30
Adish M
ccbecd1fd8 fix: remove unused ami_regions variable and update ami_region default to us-east-1 2026-02-17 15:57:01 +05:30
Adish M
726dd4f35c
Reduce volume size from 30 to 15 GB 2026-02-16 16:37:06 +05:30
Adish M
bd15c61209
Update variables.pkr.hcl 2026-02-13 17:10:48 +05:30
Adish M
1121479583
Remove 'ca-central-1' from ami_regions list 2026-02-13 14:03:32 +05:30
Adish M
fbc4d82a62
fix: update OpenSSL packages to address on EC2 AMI (#15105) 2026-01-29 16:07:13 +05:30
Adish M
39aa67aa7c
Update setup_machine.sh for PostgreSQL installation (#14909)
Removed postgresql-client from initial installation and added steps to install PostgreSQL client 14 from the official APT repository.
2026-01-05 22:24:31 +05:30
Adish M
8417c6437c Enhance setup script with improved apt reliability and retry logic 2025-12-22 12:03:31 +05:30
Akshay Sasidharan
7df27c6480 Merge remote-tracking branch 'origin/lts-3.16' into release/worfklows-sprint-14 2025-11-12 14:32:30 +05:30
Dheeraj-P-Girish
3192f61179
implemented the hubspot form tracking script for tracking the users journey (#14508)
* implemented the hubspot form tracking script for tracking the users journey while signing up with sso

* fix: removed the console logs and replaced spread oprations with object.entries

* revert submodule commit

* revert-server-submodule

* minor changes

* Removed HubSpot form and portal ID references from instance system settings

* Removed missed HubSpot SSO constants from instance settings

* Bump version to 3.20.33-lts and clean up code formatting

---------

Co-authored-by: Dheeraj P Girish <dheerajpgirish@Dheeraj-ka-MacBook-Pro.local>
Co-authored-by: gsmithun4 <gsmithun4@gmail.com>
2025-11-07 11:21:32 +05:30
Akshay
18b831f783
Refactor: Workflows with BullMQ (#14275)
* refactor(workflows): migrate core modules from Temporal to BullMQ

Update main application modules to support BullMQ-based workflow scheduling:

- Remove Temporal worker bootstrap code from main.ts
- Migrate from @nestjs/bull to @nestjs/bullmq
- Add Bull Board dashboard at /jobs with basic auth
- Register BullMQ queues in WorkflowsModule
- Add IWorkflowScheduler interface for scheduler abstraction
- Create CE stubs for WorkflowSchedulerService and ScheduleBootstrapService
- Remove workflow dependencies from AppsModule (moved to WorkflowsModule)
- Add proper route exclusion for /jobs dashboard
- Support WORKER env var for conditional processor registration

This commit completes the migration from Temporal to BullMQ for workflow
scheduling, enabling simpler deployment and better horizontal scaling.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore: remove unused Temporal imports and commented code

Clean up migration artifacts:
- Remove unused imports from main.ts (TOOLJET_EDITIONS, getImportPath, ITemporalService, getTooljetEdition)
- Remove commented TemporalService references from WorkflowsModule
- Remove temporal.service from getProviders path array
- Add missing newlines at EOF for IWorkflowScheduler.ts and schedule-bootstrap.service.ts

This cleanup prepares the codebase for complete Temporal code removal in a future commit.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* build: add BullMQ and Bull Board dependencies

Add required packages for BullMQ-based workflow scheduling:

- @nestjs/bullmq: NestJS integration for BullMQ
- @bull-board/api, @bull-board/express, @bull-board/nestjs: Queue dashboard
- bullmq: Core BullMQ library
- express-basic-auth: Authentication for Bull Board dashboard

Note: @nestjs/bull is kept for backward compatibility during migration.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore: remove Temporal service files from CE

Remove deprecated Temporal-based workflow implementation files:
- server/src/modules/workflows/interfaces/ITemporalService.ts
- server/src/modules/workflows/services/temporal.service.ts

These files are replaced by IWorkflowScheduler interface and BullMQ-based
WorkflowSchedulerService implementation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat: add comprehensive Redis configuration support for BullMQ

* refactor: remove Temporal setup and configuration from entrypoint scripts and Dockerfiles (#14294)

* refactor: remove Temporal setup and configuration from entrypoint scripts and Dockerfiles

* feat: integrate Redis support for BullMQ in preview environment

* remove worker execution logic from setup script

* Refactor: Centralise workflow execution through BullMQ (#14321)

* refactor: run all workflows through bullmq

* refactor: update imports

* chore: update subproject commit reference in server/ee

* feat: ablity to cancel workflow

* feat: implement workflow cancellation functionality with Redis support

* feat: add optional timeout parameter to requestCancellation method

* refactor: clean up formatting and add maintenance toggle event emission in AppsService

* feat: ability to cancel multiple inprogress executions

* feat: implement execution state management and display logic

* chore: update submodule commit reference

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Adish M <44204658+adishM98@users.noreply.github.com>
2025-11-05 17:54:38 +05:30
adishM98 Bot
f4565c0979 Expand default AMI regions for improved deployment flexibility 2025-11-04 15:48:52 +05:30
adishM98 Bot
cba20d168c fix(ec2-ee): Optimize AMI packaging to reduce vulnerability surface and fix production deployment 2025-11-04 15:47:57 +05:30
Adish M
618a3ff395
Fix command syntax for starting the server (#14485) 2025-11-03 10:45:15 +05:30
adishM98 Bot
30fc0fea6b remove Neo4j installation and configuration from Dockerfiles and entrypoint scripts 2025-09-12 15:54:25 +05:30
Adish M
d710f9be8f
fix: update environment variables and setup scripts for production deployment (#13925) 2025-08-26 11:40:48 +05:30
Adish M
d9000747fe
update required_plugins version for amazon to >= 1.3.2 (#13743)
* update required_plugins version for amazon to >= 1.3.2

* Add missing checkout step in Packer build workflow

* Fix required_plugins version for amazon to exact match of 1.3.2

* Fix PAT substitution in setup_machine.sh for Git configuration
2025-08-11 11:09:41 +05:30
Adish M
7796fa4377
Fix: Automating mergeing issue to lts-3.16 branch (#13700) 2025-08-06 21:11:53 +05:30
Adish M
9215def6e7
fix: AMI packer workflow and branch name correction in the ec2 setup file (#13627) 2025-08-03 13:59:12 +05:30
Adish M
ac5f3816de feat: Update ami_regions variable to include additional AWS regions 2025-07-08 17:31:07 +05:30
Adish M
ee769195c3
feat: Update Packer configuration to use Ubuntu Jammy and adjust instance types and AMI settings (#13259) 2025-07-08 16:23:03 +05:30
Adish M
ed571cafe3
feat: Update Packer workflow to checkout lts-3.6 branch and add Neo4j setup with APOC plugin (#13225)
* feat: Update Packer workflow to checkout lts-3.6 branch and add Neo4j setup with APOC plugin

* refactor: remove deprecated EC2 ce deployment files and scripts

* fix: ensure newline at end of file in setup_app and setup_machine scripts
2025-07-07 18:14:18 +05:30
Midhun G S
175345479a
Node upgrade 18->22 (#12836)
* node upgrade initial commit

* Removed node-saas and added saas. Fixed the CSS warnings from saas

* add node version 22 to dockerfile and AMI setup script

* change the base image to debian 12

* Update the node version (#12860)

* correction in builder image

* Update ce-production.Dockerfile

* upgrade node to 22.15.1

* server package lock

* package upgrade

* package upgrade

* base directory package upgrade

* revert sentry package upgrade

* fix

* added copy files

* fix dockerfile for copywrite package issue

* removed copy files package

* fix dockerfile for copywrite package issue

---------

Co-authored-by: Kavin Venkatachalam <kavin.saratha@gmail.com>
Co-authored-by: Adish M <adish.madhu@gmail.com>
Co-authored-by: Ajith KV <ajith.jaban@gmail.com>
Co-authored-by: Adish M <44204658+adishM98@users.noreply.github.com>
Co-authored-by: Muhsin Shah C P <muhsinshah21@gmail.com>
2025-06-17 13:01:46 +05:30
Adish M
ffa92c419e
Feature: Adding Modularisation for AWS Packer build (#12098) 2025-03-03 18:26:28 +05:30
Adish M
2ac17e2242
Chore: Tooljet database and marketplace are mandatory set for render (#11173)
* Chore: Tooljet database and marketplace are mandatory set on render and other deployment file for CE

* added PGRST_DB_PRE_CONFIG=postgrest.pre_config

* edited .env.example file
2024-10-30 17:31:08 +05:30
Muhsin Shah C P
53d119680f
[chore] Node-module vulnerabilities (#8226)
* started working on node upgrade to 18.18.2

* testing ci

* updated ci node version

* updated action code

* deleted all package-lock.json files

* deleted and ovverrided some packages

* deleted and fixed server & frontend vulnerabilities

* updated firestore version

* fix: ws type issue

* fix: upgraded lerna version

* regenerated package-lock.json files again

* regenerated marketplace lock file

* updated node version in other ci and docker files

* update: lock file plugin side

* updated the npm version in docker & ci files

* removed unused imports from events file

* removed dependency-review action

* updated some packages

* tried to go with current node-module of jest. had to upgrade

* fix: deprecated function usage - ts-jest

* fix: server directory lint issues

* fixed login page issue after router-dom upgrade

* updated import/no-unresolved rule to ignore import errors of  react-loading-skeleton, react-spring packages

* fix: cypress node version & package-lock issue

* regenerated cli package-lock.json

* fix: new webpack version might cause runtime issues (had issues with enterprise). lets use old version only

* fix: form-data docker issue

* removed comment
2023-12-21 11:55:35 +05:30
Adish M
c08eb83da5
Adding redis as a default deployment (#7455)
* Adding redis as a default deployment

* creating test image

* renamed redis file

* redis unit error fixed

* redis installation files fixed

* changed image in all the deployments

* removing docker automation

* changed the base image in try-tooljet dockerfile

* stopped pack build from ce

* Automation changes

* Update deployment.yaml

* Update deployment.yaml

* Update deployment.yaml

* Update deployment.yaml

* Update deployment.yaml

* Update deployment.yaml

* Update try-tooljet.Dockerfile

* Update deployment.yaml

* Update deployment.yaml

* Update deployment.yaml

* Update variables.pkr.hcl

* Update variables.pkr.hcl

* Update variables.pkr.hcl

* Update tooljet_ubuntu_focal.pkr.hcl

* Update setup_machine.sh

* Update setup_machine.sh

* Delete deploy/ec2/redis-server.service

* Update .env

* Update docker-compose.yaml

* Update docker-compose.yaml

* Update docker-compose.yaml

* Update docker-compose-db.yaml

* Update docker-compose-db.yaml

* Update .env.external.example

* Update .env.internal.example

* Update .env.internal.example

* Update tooljet-release-docker-image-build.yml

* Update tooljet-release-docker-image-build.yml
2023-09-27 11:10:52 +05:30
Adish M
531f5c8292
Packer build fix (#7301) 2023-08-21 15:27:50 +05:30
Adish M
06d3d85e88
Updating .env file (#6872) 2023-07-03 13:12:02 +05:30
Adish M
6f201cab13 Fix: release pipleline 2023-06-26 14:36:43 +05:30
Shubhendra Singh Chauhan
532a77a8cf
Merge branch 'develop' into release/marketplace_1.1 2023-06-22 11:53:13 +05:30
Adish M
06c454304f
Update setup_machine.sh 2023-06-21 11:45:11 +05:30
Akshay
1602317ad5
Add support for oracle 9i db (#6637)
* update oracledb package for node 18

* add oracle instant client 11 and 21 binaries for linux

* Revert "add oracle instant client 11 and 21 binaries for linux"

This reverts commit 9ba7d18685.

* add ability to specify instant client version

* update dockerfiles to include oracle 11 instant client

* update ami build for oracle instant client 11

* add data migration for oracle ds default client version 21

* remove console.log in code

* fix server dev dockerfile

* define LD_LIBRARY_PATH

* fix LD_LIBRARY_PATH assignment
2023-06-21 11:39:30 +05:30
Adish M
637028a075
Update setup_machine.sh 2023-06-21 10:56:26 +05:30
Midhun G S
88549c15c6
building fe assets synchronously (#6514) 2023-05-24 22:26:24 +05:30
Adish M
202bcda84a
Changes in ec2 build-packer (#5590)
* building image for marketplace

* packer file changes

* changes in ec2 build-packer

* reverted changes
2023-02-22 14:38:40 +05:30
Kavin Venkatachalam
1efcec9e09
[Feature]: Update Node from 14 to 18 (#5393)
* Updated package.json to support node 18

* Updated package-lock.json with force install

* Updated frontend package.json

* Updated server package.json

* update frontend package-lock.json

* update render dockerfile

* update dockerfiles for node 18

* update node version in actions

* fix setup script for AMI

* install using nvm

* debug

* source bashrc

* change ami os version

* change base ami

* make use of ubuntu focal lts version

* update script

* change branch name

* revert branch name

* update workflow

* node upgrade [plugins] modules updated (#5482)

* modules updated

* adds saphana and oracle package

* adds saphana and oracle package

* eslint rules for -unresolved and resolved properties

---------

Co-authored-by: Akshay Sasidharan <akshaysasidharan93@gmail.com>
Co-authored-by: Arpit <arpitnath42@gmail.com>
2023-02-06 17:38:44 +05:30
Adish M
d5775e20e9
Added workflow dispatch (#5312)
* Added workflow dispatch

* adding Tooljet_db_* vars
2023-01-12 14:12:45 +05:30
Adish M
e0fe9a0cf9
AMI fix for v2 (#5311) 2023-01-12 12:44:59 +05:30
Akshay
2ce4fa5d64
Chore: Update AMI for v2 (#5225)
* update ami setup for v2

* revert branch name
2023-01-09 14:57:48 +05:30
Mehdi El Aouni
d3e6a201f4
Adding database name to the psql query (#3711)
TO solve "Can't connect to the database. Kindly check the credenials provided in the .env file!" error
2022-08-03 12:48:41 +05:30
Akshay
a13a2d4d63
Chore: ToolJet setup to try locally (#3158)
* add dockerfile for try tooljet setup

* update npm scripts and logged messages

* refactor npm scripts for dist folder scripts

* update usages of db setup and seed

* setup script to init db and boot app

* fix dockerfile

* update dockerfile

* user production.Dockerfile as base image

* fix script

* fix script exit

* fix lint
2022-05-31 18:42:36 +05:30
Akshay
bd56792d58
Add deployment platform for telemetry (#3126)
* add deployment platform telemetry

* make use of config service
2022-05-26 07:11:56 +05:30
Shubhendra Singh Chauhan
eb2b2759b8
[docs] renamed deployment directory to Setup (#2847) 2022-04-19 08:34:54 +05:30
Kahlil Abreu
7426e71930
Add ap-east-1 to the list of AWS Regions for AMI release (#2657) 2022-03-29 10:30:47 +05:30
Muhsin Shah C P
eaf93616ae
Feature/oracledb plugin (#2388)
* oracledb initial commit
- Added oracledb package

* Implemented connection and operations

* Working on oracle db integration

* Implemented query & bulk operations

* Fixed initial client reintialization problem

* Added custom client library path option

* Added oracle client library installation steps to dev docker file

* Testing changed docker setup on cloud

* Testing heroku app.json

* Removed unwanted scripts from dockerfiles

* Updated field label

* update oracledb client lib for ec2

* update package-lock

* refactoring code

* Updated docs

* Resolved some lint errors

Co-authored-by: Akshay Sasidharan <akshaysasidharan93@gmail.com>
2022-03-15 08:00:02 +05:30
Gandharv
2f3b441c0a
Move plugins to root (#1728)
* feat: move plugins to root

* modify tsconfig

* add .gitignore

* delete old plugins file

* add parcel

* docker compose volume mount

* add gcs

* add typescript to plugins folder

* gcs to ts

* add dynamodb

* add elastic search

* add firestore

* add gsheets

* add graphql

* add mongodb

* mssql

* add mysql

* add postgresql

* add redis

* add s3

* add slack

* add stripe

* remove plugin related packages from pkgjson

* add lib folder

* add gitignore

* remove typescript generated files

* remove generated file

* remove generated files

* add twilio

* add dist to docker compose cache binding

* add dist prefix

* cleanup - 1

* delete dist

* rename to index.ts + add jest config

* add it.todo in tests

* test fixes

* test file changes

* fix type checks

* add @tooljet/plugins to server package json

* esm vs commonjs bug, reduce got to 11.8.2 from 12.0.0

* docker file npm package version fix

* add typesense

* cleaup - 2

* add sendgrid

* add lerna build and clean script for all packages + tsconfig

* cleanup -3

* add plugins build step

* add missing plugins build step in npm run build

* add mssql, mysql & postgres as singleton classes

* add db connection to cache only if datasourceId is available

* client: add data source schema/manifest files

* add query operations files

* logic for wrapping form with schema

* add script to create index file

* add @tooljet/plugins to frontend folder

* cleanup 1 -frontend

* cleanup - 2 // frontend // data queries

* add client and index to gitignore

* update gitignore

* fix lint & test

* update ci

* fix unit, e2e

* cleanup -3

* fix test

* fix tests

* fix indent

* try npm ci

* fix tests

* fix typo

* fix

* rename file for server entry

* heroku fix

* add main and types entry points in pkg json

* move common to root

* cleanup - 4: remove redundant $ sign prefix

* cleanup - 4: remove redundant $ sign prefix

* update options in-sync before DOM is painted

* change type cloud to cloud storage

* update readme

* update ci.yml

* update ci yml

* add pkg-lock.json

* rename index.ts to server.ts

* update lock files

* add server package.lock

* remove unused import

* revert commit: add minio

* add root dep

* import server.ts

* remove plugins build step

* add npm shrinkwrap

* update version - plugins

* add new version - 0.0.8

* upgrade version

* move to symlinked package

* add lock file

* feat: add icon inside package

* add plugin creation docs

* Remove seed

* move icons to plugins folder

* install pg dep

* add react to packages

* add seed cmd

* revert change

* add plugins build in lint, e2e, unit

* e2e, lint use npm ci

* update dockerfile for plugins

* try combining release with web

* limit memory on release

* try executing seed script post transpile

* try executing seed from server directory

* update seed execution

* add minio

* add correct type

* add minio to pkg json

* remove old file

* fix provider key

* add python installable + npm ^7.2.0 (#1752)

* add python installable + npm ^7.2.0

* add py to prod file

* pin npm version to 7.20.0

* pin npm version to 7.20.0

* split into multi stage build and remove python for buildx

* copy plugins from buider stage

* update dependencies

* add freetds dependency

* update server dockerfile

* update client dockerfile

* update dev dockerfile and compose file

* fix entrypoint

* fix server dev dockerfile

* update docker-compose

* remove npm install on root dir on docker build

* fix heroku script

* make lerna prod dependency to enable prod builds

* remove redundant env setup

Co-authored-by: Akshay Sasidharan <akshaysasidharan93@gmail.com>
Co-authored-by: navaneeth <navaneethpk@outlook.com>
2022-01-17 12:38:17 +05:30
Akshay
4f8697a7d2
make ami public by default (#1400) 2021-11-15 08:18:38 +05:30
Akshay
a3d009236c
Revert "Uses double quotes to expand expressions (#1000)" (#1180)
This reverts commit 2d5af45309.
2021-10-21 13:02:55 +05:30