Commit graph

32 commits

Author SHA1 Message Date
Midhun G S
1aa477fc31
[v2-beta] migration improvement (#5047)
* migration improvement

* fix

* added constants and comments
2022-12-27 16:16:13 +05:30
Akshay
602b1b4ad3
Feature: Tooljet Database (#4951)
* wip

* internal db per workspace

* fix async query

* feat: add storage layer route

* feat: add drawer component

* feat: add react-table to load data

* feat: add columns form

* feat: add create column form, create row form

* feat: add postgrest js

* add tooljet db controller to proxy requests to postgrest

* util: add postgrest filter builder helper utility

* feat: add filter popover

* use helper utility for building query

* add sortable filters

* add box shadow for filter popup

* use overlay trigger

* use react select

* add new column addition

* add dropdown for table header, table list

* Move filter.jsx

* feat: add sort popover

* feat: add postgrest js .order fn

* setup tooljetdb with restricted grants for users

* make db schemas added loaded dynamically on postgrest server

* fix query

* sign jwt token to auth user at postgrest

* update db schema user with workspace

* chore: add table listing

* update data and columns from api

* feat: add context api for sharing data

* add ability to create table, view tables and add columns

* use columns for sort from context api

* fix ormconfig

* feat: add table listing integration

* feat: add create table integration

* fix for rds deployment

* add internal table translation instead of schema

* remove tooljetdb as a datasource

* wrap placeholder on proxy query

* add active workspace guard

* scope tooljetdb by workspace

* update active workspace guard

* seperate proxy related concerns to different service

* make use of org id param

* rename storage layer to tooljet databse

* update specs

* feat: Update list when new table added

* feat: add create column

* chore: add orgId to url + misc changes

* chore: move popover to separate file

* remove unused var

* rename files

* feat: add multiple columns

* feat: add new row

* removes postgrest-js from pkg lock

* feat: add row data

* feat: add sorting

* feat: allow row deletion

* feat: add search

* feat: add filtering

* feat: add edit mode

* feat: add columns while edit table

* add view table action

* update setup for column constraint

* fix query

* integrate view_table, primary key field

* render toogle for boolean data type

* update view table query for primary key

* fetch metadata refactor

* add capability to set default values

* feat: allow deletion of record based on primary key

* feat: add default value while creating column

* send query from sort & filter component

* css changes

* allow empty data

* add requested changes

* add err message

* add common fn

* allow sort + filter

* remove unwanted defaults key

* css changes

* add more operators

* dark mode fixes

* add drawer footer

* add loader for list tables

* add dashboard design changes

* design changes

* add capability to drop table and delete column

* add breadcrumbs

* design changes

* add profile

* refactor tooljetdb controller

* update routes

* add empty page changes

* delete column fix

* fix delete column

* design changes

* fetch tables post delete

* homepage changes

* hide ellipsis on hover

* add org settings page

* add edit + create org

* add notification center

* fix: group permissions switch issue

* add logo

* remove anchor tag

* fix merge conflicts

* css changes

* add err boundary

* setup query editor

* css changes

* fix: merge conflicts

* add menuPortal prop to filterform and sort form

* fix seed

* fix: build

* design changes

* design changes + refactor code

* fix imports

* fix: drawer issue on delete table

* add search box changes

* fix: tablename max-length 255

* fix: set newly created table as selected item

* remove edit column option

* added badges to enterprise only features

* disable edit column

* table styles

* fix: popover position, placeholder default

* fix: display boolean values in table

* fix: tooljet database default type values

* css changes

* add query manager for tooljet db with create and list row

* dark mode fixes

* remove Header component

* add ability to delete tooljetdb rows from query manager

* add ability to update tooljetdb rows

* dark mode fixes

* css changes

* display actions icon on hover

* folder onclick change

* add empty page styles

* fix proxy requests

* feat: randomize icon creation

* add max items per page prop for pagination

* removes unwanted position attr

* add table name validation + disable submit btn while api fetch

* [Bugfix] internal storage toast  | trigger toasts for running preview db queries (#5019)

* resolves: no toasts are fired when preview query is run for db queries

* fire success toast for created and no content status text for query success

* remove invalid migration

* skip migration if tooljet db already created

* fix: app clone icon param

* fix: show confirmation box if filter options are empty in query (#5021)

* for now: show native confirmation box of the brower to confirm the delete all query

* typo

* Revert "typo"

This reverts commit b5ce5ed889.

* cleaned

* cleaned

* show confirmation box if filter options are empty in query

* [Refactor/Bugfix] database query (#5028)

* refactored list rows operations

* remove unwanted cls

* refactor create row

* reafactored update rows

* refactored delete rows

* padding fix for tj-query

* add static templates

* review changes

* remove unused file

* Chore: tooljetdb render setup (#5033)

* add postgrest for render preview deploy

* pin version

* add healthCheckPath

* remove health check

* handle database url parsing db params

* add defaults for tooljetdb env

* fix hostname

* handle env in migration files

* refactor dbconfig build

* fix pg db usage

* add parsed env context

* add tooljetdb env

* refactor db config utils

Co-authored-by: gandharv <gandharvkumargarg@gmail.com>
Co-authored-by: Shubhendra <withshubh@gmail.com>
Co-authored-by: Arpit <arpitnath42@gmail.com>
2022-12-23 02:09:57 +05:30
Midhun G S
a60706e80f
fix for app preview error (#4994) 2022-12-15 16:01:10 +05:30
Midhun G S
487252c5ea
Feature - Multi Environment support (#4833)
* initial commit

* multi env changes

* multi-env changes

* entity fixes

* data query changes

* fix

* trying to avoid conflict with EE

* moved version creation to app creation function

* fixing some issues

* execution of data query

* revert options changes

* changed migration

* fixed some migration issues: testing migration

* multi env support

* app import export fix

* fixes

* migration fixes

* removed plugins from data query

* fixing some migration issues

* fixes

* remove console log

* fix

* front end api changes

* backward compatibility for app import

* Fixed a bug

* correcting some mistakes

* Added constraints and fixed some issues

* changes

* fix for data source listing

* fixing version operation issues

* remove kind from data query

* removed kind from data query

* fixes

* fixes

* fix for version creation

* migration fixes

* Fixed preview and run query issues

* Fix: new version and event query id issue

* fixed rest api oauth issue
- next test refresh token

* import export changes

* fixes for app import

* import fix

* added await for for loops

* fix

* fix for migration

* Fixed backend oauth-envId issue

* import export changes

* migration fixes

* fix

* fix

* fix for app import from 0.9.0

* test case fixes

* test case fixes

* making app name mandatory for import

* adding type for options

* fix: imported apps query linking issues

* review changes

* lint issue fixes

* added on delete cascade

Co-authored-by: Muhsin Shah <muhsinshah21@gmail.com>
2022-12-09 20:23:42 +05:30
Gandharv
a1fd1fc301
[Feature] Make plugins installable (#3069)
* feat: add user avatar

* update: @nest/platform-express from 8.0.0 to 8.4.4

* add avatar_id in login response

* add user avatar upload in frontend

* align cross divider with layout icons'

* generate nest model - extensions

* Add extensions module

* Add extension to datasouce

* add not implemented check

* create extension

* refactor

* cleanup

* fix tests

* reduce the avatar size on homepage

* poc: run js code from string

* resolve conflicts

* fix conflicts

* add globals

* add new route

* add icon, manifest file upload

* complete user flow

* add flow for data queries

* add dynamic manifest instead of local datasource types

* add version attr

* remove unused code

* add version

* rename extension(s) -> plugins(s)

* add test connection method

* feat: add marketplace listing page

* Add install plugin cmd + missing attrs {name, repo, desc} to plugin

* add missing icon

* - Add npm workspaces for marketplace monorepo
- Added cassandra datasource plugin
- Created upload to s3 script
- Created plugins.json entry file

* install plugin from s3 bucket

* cleanup

* update pkg locks

* fix icon render

* cleanup

* marketplace changes

* ui changes

* operations file load fix + revert vm2

* update module from string to 3.2.1

* load plugins.json from local file instead of remote

* install plugin from local file if not production environment

* add sqlite

* feat: add plivo api plugin

* exp: add heroku 22 stack

* update assets include path

* Revert "exp: add heroku 22 stack"

This reverts commit a8926b36e1.

* add integrations link

* Add casl ability for plugin

* load host from env else fallback to default

* update imports

* remove sqlite

* typo

* add marketplace flag to cli command

* move ts and ncc to devDep

* add hygen templates for marketplace

* cli tree-node path fix

* template indent fix

* TOOLJET_URL -> MARKETPLACE_TOOLJET_URL

* add tests

* refactor: move to plugins.helper for get-service helper utility

* fix; typo

* update package-lock.json

* review changes

* remove a href

* remove bg color + redirect issue due to href

* add test url

* fix crash on search

* remove extra slash

* feat: allow plugin to be installed from github repository

* remove unwanted args from cli command

* add repo attr while save

* feat: add feature toggle for marketplace feature

* fix: make default config as false

* chore: remove hyperlink

* fix: failing build

* chore: update s3 url to point to prod

* fix failing test

* fix test

* fix: test case

* update module from string pkg

* update env

* fix test

* fix test

* add readme file

* Update README.md

Co-authored-by: Akshay Sasidharan <akshaysasidharan93@gmail.com>
2022-10-27 16:59:43 +05:30
Midhun G S
c862db0f0b
merge main to develop (#4527)
* [ Hotfix ] :: App breaking while reconnecting sheets (#4514)

* app breaking while reconnecting sheets

* back-end fix

* open api fix

Co-authored-by: gsmithun4 <gsmithun4@gmail.com>

* Bump to v1.27.2

Co-authored-by: Kiran Ashok <stepinfwd@gmail.com>
2022-10-19 20:04:57 +05:30
Sherfin Shamsudeen
a8283831d1 Merge branch 'main' into develop 2022-10-14 15:02:00 +05:30
Muhsin Shah C P
2d777c2d51
[hotfix] Fixed oauth refresh token issue (#4406)
* Fixed refresh token issue

* Added re-auth feature

* Fixed token issue

* fixes

* Added public app check

Co-authored-by: gsmithun4 <gsmithun4@gmail.com>
2022-10-14 13:25:32 +05:30
Arpit
74beb544fe
[Feature/plugin] Zendesk integration 🚀 (#3943)
* plugin:init

* zendesk svg icon

* zendesk integration 🚀

* clean up

* unit tests

* docs: zendesk

* resolves: error proof

* fix typos
2022-10-06 13:45:42 +05:30
Muhsin Shah C P
638fcf7223
[Feature] Now multiple users can login through RestAPI-Oauth 🚀 (#4052)
* Implemented multiple access token feature
- working on refresh token case

* worked on refresh token flow

* added multiple token ability to openapi plugin too
- fixed some bugs

* changed tokenData to token_data
- fixed some issues

* added user type with user id

* changed user type

* Rewrote some function
- added switch for enabling and disable multi auth
- fixed some bugs
- refactored the code

* fixed view app issue

* Fixed public app issue

* cleaning some code

* (public_app) add a check to avoid oauth login redirect when there is no access token

* reverted all changes of openapi (temporary)
- will add multi token functionality once done with restapi

* fixed a bug

* fixed a bug

* refactored some code

* changed the switch text

* pr changes

* changed token_data back to tokenData

* cleaning code

* removed token data from datasources query

* removed some lines

* added a comment
2022-09-19 20:27:37 +05:30
Midhun G S
0545528504
Support for multiple workspaces 🚀 (#2778)
* multi org changes

* Initial changes

* changes

* manage sso page

* Multi-organization changes

* Multi organization changes

* multi-org changes

* multi-org changes

* multi-org changes

* multi-org fixes

* env variables app.json changes

* multi-org-fix

* user invitation token fix

* multi-org group permission fix

* multi-org app privilege

* google oauth fix

* Remove enable signup for form login

* Multi organization fixes

* multi-org user invite flow changes

* multi-org sign up fix

* rebase and multi-org fixes

* revert testing logs

* test logs revert

* migration changes

* migration file fix

* error message changes

* git login for private email fix

* dropdown fix

* test cases

* e2e test cases added

* test cases fix

* documentation changes

* testcases fix

* testcases added

* replace findOne with findOneOrFail

* accept invite testcases

* login page fixes

* added encrypted tag

* review comments

* migration fixes

* improvements

* manage sso loading fix

* review comments

* migration file changes

* new organization creation bug fix

* added e2e testcases

* added testcases

* Update data_sources.controller.ts
2022-05-05 12:38:42 +05:30
Muhsin Shah C P
722a701b22
[Feature] option to delete a data source 🚀 (#2790)
* Implemented frontend
- icon and confirmation modal
- delete fn in services

* Implemented backend Endpoint
- Changed confirmation modal msg

* Added unit test for delete operation

* Changed the modal msg again

* Fixed delete btn alignment issue

* Resolved a bug

* changed popover width

* Now delete endpoint will only return the status

* Added a new spec to test delete a data-source from a specific version

* Added deleteDataSource ability

* Removed commented code

* Added BadRequestException in else case

* Resolved an error

* Changed confirmation modal
2022-05-02 18:42:14 +05:30
Muhsin Shah C P
05edf4bcf0
Feature: Added refresh token functionality to Rest API OAuth flow (#2536)
* Working on refresh token fn
- Added refresh token to options

* Added and tested google oauth refresh token flow

* Working on outlook refresh token code

* Tested refresh token fn with google oauth and outlook

* Resolving a bug

* Added prompt=consent to auth url

* Appended custom params to authUrl

* Resolved a test error
2022-03-25 12:48:29 +05:30
Akshay
59cf3b1931
Fix: Googlesheets OAuth refresh token flow (#2118)
* implement google sheet oauh token refresh flow

* format lint

* make use of constructor name as instanceof fails

* remove unused import

* force consent to get refresh token on googlesheets ds
2022-02-05 07:09:40 +05:30
Gandharv
2f3b441c0a
Move plugins to root (#1728)
* feat: move plugins to root

* modify tsconfig

* add .gitignore

* delete old plugins file

* add parcel

* docker compose volume mount

* add gcs

* add typescript to plugins folder

* gcs to ts

* add dynamodb

* add elastic search

* add firestore

* add gsheets

* add graphql

* add mongodb

* mssql

* add mysql

* add postgresql

* add redis

* add s3

* add slack

* add stripe

* remove plugin related packages from pkgjson

* add lib folder

* add gitignore

* remove typescript generated files

* remove generated file

* remove generated files

* add twilio

* add dist to docker compose cache binding

* add dist prefix

* cleanup - 1

* delete dist

* rename to index.ts + add jest config

* add it.todo in tests

* test fixes

* test file changes

* fix type checks

* add @tooljet/plugins to server package json

* esm vs commonjs bug, reduce got to 11.8.2 from 12.0.0

* docker file npm package version fix

* add typesense

* cleaup - 2

* add sendgrid

* add lerna build and clean script for all packages + tsconfig

* cleanup -3

* add plugins build step

* add missing plugins build step in npm run build

* add mssql, mysql & postgres as singleton classes

* add db connection to cache only if datasourceId is available

* client: add data source schema/manifest files

* add query operations files

* logic for wrapping form with schema

* add script to create index file

* add @tooljet/plugins to frontend folder

* cleanup 1 -frontend

* cleanup - 2 // frontend // data queries

* add client and index to gitignore

* update gitignore

* fix lint & test

* update ci

* fix unit, e2e

* cleanup -3

* fix test

* fix tests

* fix indent

* try npm ci

* fix tests

* fix typo

* fix

* rename file for server entry

* heroku fix

* add main and types entry points in pkg json

* move common to root

* cleanup - 4: remove redundant $ sign prefix

* cleanup - 4: remove redundant $ sign prefix

* update options in-sync before DOM is painted

* change type cloud to cloud storage

* update readme

* update ci.yml

* update ci yml

* add pkg-lock.json

* rename index.ts to server.ts

* update lock files

* add server package.lock

* remove unused import

* revert commit: add minio

* add root dep

* import server.ts

* remove plugins build step

* add npm shrinkwrap

* update version - plugins

* add new version - 0.0.8

* upgrade version

* move to symlinked package

* add lock file

* feat: add icon inside package

* add plugin creation docs

* Remove seed

* move icons to plugins folder

* install pg dep

* add react to packages

* add seed cmd

* revert change

* add plugins build in lint, e2e, unit

* e2e, lint use npm ci

* update dockerfile for plugins

* try combining release with web

* limit memory on release

* try executing seed script post transpile

* try executing seed from server directory

* update seed execution

* add minio

* add correct type

* add minio to pkg json

* remove old file

* fix provider key

* add python installable + npm ^7.2.0 (#1752)

* add python installable + npm ^7.2.0

* add py to prod file

* pin npm version to 7.20.0

* pin npm version to 7.20.0

* split into multi stage build and remove python for buildx

* copy plugins from buider stage

* update dependencies

* add freetds dependency

* update server dockerfile

* update client dockerfile

* update dev dockerfile and compose file

* fix entrypoint

* fix server dev dockerfile

* update docker-compose

* remove npm install on root dir on docker build

* fix heroku script

* make lerna prod dependency to enable prod builds

* remove redundant env setup

Co-authored-by: Akshay Sasidharan <akshaysasidharan93@gmail.com>
Co-authored-by: navaneeth <navaneethpk@outlook.com>
2022-01-17 12:38:17 +05:30
Akshay
823833b12e
Feature: Versioning data sources & queries (#1620)
* add migrations for versioning datasources and dataqueries

* eslint migrations folder

* update ormconfig to use app context in migrations

* make app to make use versioned data sources and queries in editor

* app import should honor data source and query versioning

* register ts-node path on running migrations

* load data queries when finding versions

* update e2e tests

* handle for invalid oauth2 datasources

* update e2e tests

* introduce db migrations access lock to handle concurrent backfill

* scope migrations by missing version id

* fix import export without data source/query versions

* fix import

* fix spec

* handle invalid data on migration

* fix import

* remove console logs

* handle new app imports

* fix version check

* fix async creation

* explicitly add timestamps on version import
2022-01-04 13:34:12 +05:30
Arpit
26c9cc655c
Fix linting errors across the app (#785)
* eslint-setup: rules for frontend and server

* setup pre-commit:hook

* frontend:eslint fixes

* frontend eslint errors and warning fixed

* eslint:fix for ./server

* fix server/test: expectatin string lint/error

* pre-commit:updated

* removed unwanted install cmd from docker file

* recommended settings and extension for vscode

* husky prepare script added

* updated extension recommendations

* added prettier as recommended extension

* added pre-commit to package.json

* remove .prettierrc file

* resolve changes

* resolve changes
2021-09-21 19:18:28 +05:30
Akshay
bc193c081f
Fix: Only update credentials when data source is updated (#763)
* only update credentials when data source is updated

* fix spec
2021-09-21 10:20:02 +05:30
navaneeth
12d8c4af28 Fix for REST API query previews returning 403 2021-08-30 23:36:05 +05:30
navaneeth
116fc1f9f3 Feature: OAuth support for REST API sources 2021-07-25 22:50:19 +05:30
navaneeth
14b2114424 Fix bug that prevented datasources from being updated 2021-07-25 14:25:31 +05:30
navaneeth
50ef839c8b Auto-load plugins from plugins directory 2021-07-25 11:39:34 +05:30
navaneeth
35388d8930 Policies and tests for query endpoints 2021-07-24 23:39:25 +05:30
navaneeth
9a1712fb7f Slack OAuth integration 2021-07-24 14:47:38 +05:30
navaneeth
85f97e5e63 Testing datasource connections 2021-07-18 12:28:05 +05:30
navaneeth
1ae00756a8 Endpoint to fetch oauth auth url 2021-07-17 23:24:14 +05:30
navaneeth
52253f26ad Feature: update data source connection options 2021-07-16 20:33:20 +05:30
navaneeth
2fb63ed7e4 DynamoDB as datasource 2021-07-16 18:36:19 +05:30
navaneeth
2dfb3f78c8 Feature: update datasources 2021-07-16 17:40:10 +05:30
navaneeth
b2adb9144e Running queries + firestore query service 🚀 2021-07-14 19:44:35 +05:30
navaneeth
f0ead25571 Endpoint: create datasource with encrypted credentials 2021-07-12 16:06:53 +05:30
navaneeth
49a7a7451b endpoint for app data sources 2021-07-11 14:09:25 +05:30