# Introduction
Followup https://github.com/twentyhq/twenty/pull/16863
Important note: This is not an upgrade command and will have to be
manually run
In this pull request we're introducing a coding that will allow this
[migration](https://github.com/twentyhq/twenty/blob/clean-orphan-metadata/packages/twenty-server/src/database/typeorm/core/migrations/utils/1767002571103-addWorkspaceForeignKeys.util.ts#L3)
to pass, it enforces the `workspaceId` foreignKey on all metadata
entities. Allowing workspace deletion cascading of all its related
entities and avoiding orphan metadata entities to reoccur in the future
Also introduced a small migration that will set the workspaceId col type
to `uuid`, as it has been historically `varchar`
This migration is a requirement for the above command to work
successfully
## Note
Chunking by relations fields the orphan field deletion as would take way
too much time within a transac,
## Test
Tested on a prod extract locally ( both dry and not dry )
# Introduction
Related to https://github.com/twentyhq/core-team-issues/issues/1995
This PR introduces the basis of the `twentyStandard` application as code
on demand, it's highly tied to `ids` where it will becomes workspace
agnostic following the builder and runner `universalIdentifier` refactor
later.
The goal here to allow computing the `allFlatEntityMaps` `to` of the
`twentyStandard` application on a empty workspace ( workspace creation
). Allowing installing the twenty standard app through a workspace
migration instead of passing by the sync metadata
Nothing done will be run in production for the moment if it's not the
small validation refactor we've introduced
Please note that everything introduced here will be replaced at some
point by a twenty app instance when the twenty sdk is mature enough to
handle of the edge cases we need here
## How we've proceeded
We've been iterating over every workspace entity both objects and their
fields, and transpiled them to flatEntity.
Being sure we migrate the defaultValue, settings and so on accordingly.
We've also compute all the ids in prior of the whole entities
computation so we don't face any hoisting issue.
## Current state
At the moment only handling all of the 29 standard objects and their
fields
Settings a unique universalIdentifier for all of them
Will come views, agent role targets and so on later
## `workspace:compute-twenty-standard-migration` command
This command allow generating a workspace migration that will result in
installing the twenty standard app in an empty workspace
It's temporary and aims to allow debugging for the moment we might not
keep it in the future as it is right now
It contains debug writeFileSync which is expected no worries greptile
## `LabelFieldMetadataIdentifierId`
Small refactor allowing defining the label identifier field metadata id
of a uuid field metadata type for system object, as some of our standard
object don't have a name field and don't aim to
Also please note that we might remove this build options later in the
sake of the currently installed universal identifier application that we
could compare with the deterministic twenty standard one
## `runFlatFieldMetadataValidators`
Deprecated this pattern which was redundant and not v2 friendly pattern
## Current errors that will address in upcoming PR
Current standard objects and fields metadata does not pass the
validation that we have in place, as historically the sync metadata
would directly consume the repositories and would just ignore the
validation. This is about to change.
Will handle the below errors in dedicated PRs as they will required
upgrade commands in order to migrate the data, or will handle that from
the sync metadata instead still to be determined but nothing critical
here
- camel case field metadata name
- options label invalid format
```json
{
"status": "fail",
"report": {
"fieldMetadata": [
{
"status": "fail",
"errors": [
{
"code": "INVALID_FIELD_INPUT",
"message": "Name should be in camelCase",
"userFriendlyMessage": {
"id": "P+jdmX",
"message": "Name should be in camelCase"
},
"value": "iCalUID"
}
],
"flatEntityMinimalInformation": {
"id": "68dd83cd-92c8-4233-bb28-47939bab6124",
"name": "iCalUID",
"objectMetadataId": "11c16ab6-9176-439e-a2db-a12c5a58a524"
},
"type": "create_field"
},
{
"status": "fail",
"errors": [
{
"code": "INVALID_FIELD_INPUT",
"message": "Value must be in UPPER_CASE and follow snake_case \"email\"",
"userFriendlyMessage": {
"id": "UBPzFQ",
"message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
"values": {
"sanitizedValue": "email"
}
},
"value": "email"
},
{
"code": "INVALID_FIELD_INPUT",
"message": "Value must be in UPPER_CASE and follow snake_case \"sms\"",
"userFriendlyMessage": {
"id": "UBPzFQ",
"message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
"values": {
"sanitizedValue": "sms"
}
},
"value": "sms"
}
],
"flatEntityMinimalInformation": {
"id": "e3caaf2a-e07d-4146-8dfc-9eef904e82c9",
"name": "type",
"objectMetadataId": "4b777de5-4c7b-4af4-9b92-655c0f87512b"
},
"type": "create_field"
},
{
"status": "fail",
"errors": [
{
"code": "INVALID_FIELD_INPUT",
"message": "Value must be in UPPER_CASE and follow snake_case \"incoming\"",
"userFriendlyMessage": {
"id": "UBPzFQ",
"message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
"values": {
"sanitizedValue": "incoming"
}
},
"value": "incoming"
},
{
"code": "INVALID_FIELD_INPUT",
"message": "Value must be in UPPER_CASE and follow snake_case \"outgoing\"",
"userFriendlyMessage": {
"id": "UBPzFQ",
"message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
"values": {
"sanitizedValue": "outgoing"
}
},
"value": "outgoing"
}
],
"flatEntityMinimalInformation": {
"id": "d96233a4-93be-45ea-9548-3b50f3c700cf",
"name": "direction",
"objectMetadataId": "480a648a-d2e5-482a-992f-ef053e1b4bb0"
},
"type": "create_field"
},
{
"status": "fail",
"errors": [
{
"code": "INVALID_FIELD_INPUT",
"message": "Value must be in UPPER_CASE and follow snake_case \"from\"",
"userFriendlyMessage": {
"id": "UBPzFQ",
"message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
"values": {
"sanitizedValue": "from"
}
},
"value": "from"
},
{
"code": "INVALID_FIELD_INPUT",
"message": "Value must be in UPPER_CASE and follow snake_case \"to\"",
"userFriendlyMessage": {
"id": "UBPzFQ",
"message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
"values": {
"sanitizedValue": "to"
}
},
"value": "to"
},
{
"code": "INVALID_FIELD_INPUT",
"message": "Value must be in UPPER_CASE and follow snake_case \"cc\"",
"userFriendlyMessage": {
"id": "UBPzFQ",
"message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
"values": {
"sanitizedValue": "cc"
}
},
"value": "cc"
},
{
"code": "INVALID_FIELD_INPUT",
"message": "Value must be in UPPER_CASE and follow snake_case \"bcc\"",
"userFriendlyMessage": {
"id": "UBPzFQ",
"message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
"values": {
"sanitizedValue": "bcc"
}
},
"value": "bcc"
}
],
"flatEntityMinimalInformation": {
"id": "961c598e-67c3-452d-8bb2-b92c0bc64404",
"name": "role",
"objectMetadataId": "8af8a13c-ff97-4cd3-b70d-52a7dc2924b4"
},
"type": "create_field"
},
{
"status": "fail",
"errors": [
{
"code": "INVALID_FIELD_INPUT",
"message": "Label must not contain a comma",
"userFriendlyMessage": {
"id": "k731jp",
"message": "Label must not contain a comma"
},
"value": "Commas and dot (1,234.56)"
},
{
"code": "INVALID_FIELD_INPUT",
"message": "Label must not contain a comma",
"userFriendlyMessage": {
"id": "k731jp",
"message": "Label must not contain a comma"
},
"value": "Spaces and comma (1 234,56)"
},
{
"code": "INVALID_FIELD_INPUT",
"message": "Label must not contain a comma",
"userFriendlyMessage": {
"id": "k731jp",
"message": "Label must not contain a comma"
},
"value": "Dots and comma (1.234,56)"
}
],
"flatEntityMinimalInformation": {
"id": "7fa20caf-2597-42e3-84e5-15a91b125b9b",
"name": "numberFormat",
"objectMetadataId": "a6974302-9e72-461c-aa09-9390f4ff16fc"
},
"type": "create_field"
}
],
"objectMetadata": [],
"view": [],
"viewField": [],
"viewGroup": [],
"index": [],
"serverlessFunction": [],
"cronTrigger": [],
"databaseEventTrigger": [],
"routeTrigger": [],
"viewFilter": [],
"role": [],
"roleTarget": [],
"agent": []
}
}
```
**Before**
- any user with workpace_members permission was able to remove a user
from their workspace. This triggered the deletion of workspaceMember +
of userWorkspace, but did not delete the user (even if they had no
workspace left) nor the roleTarget (acts as junction between role and
userWorkspace) which was left with a userWorkspaceId pointing to
nothing. This is because roleTarget points to userWorkspaceId but the
foreign key constraint was not implemented
- any user could delete their own account. This triggered the deletion
of all their workspaceMembers, but not of their userWorkspace nor their
user nor the roleTarget --> we have orphaned userWorkspace, not
technically but product wise - a userWorkspace without a workspaceMember
does not make sense
So the problems are
- we have some roleTargets pointing to non-existing userWorkspaceId
(which caused https://github.com/twentyhq/twenty/issues/14608 )
- we have userWorkspaces that should not exist and that have no
workspaceMember counterpart
- it is not possible for a user to leave a workspace by themselves, they
can only leave all workspaces at once, except if they are being removed
from the workspace by another user
**Now**
- if a user has multiple workspaces, they are given the possibility to
leave one workspace while remaining in the others (we show two buttons:
Leave workspace and Delete account buttons). if a user has just one
workspace, they only see Delete account
- when a user leaves a workspace, we delete their workspaceMember,
userWorkspace and roleTarget. If they don't belong to any other
workspace we also soft-delete their user
- soft-deleted users get hard deleted after 30 days thanks to a cron
- we have two commands to clean the orphans roleTarget and userWorkspace
(TODO: query db to see how many must be run)
**Next**
- once the commands have been run, we can implement and introduce the
foreign key constraint on roleTarget
Fixes https://github.com/twentyhq/twenty/issues/14608
Closes#14726
### Added
- `trashRetentionDays` field to workspace entity (default: 14 days)
- Automated trash cleanup using BullMQ jobs
- Daily cron (00:10 UTC) that enqueues cleanup jobs for all active
workspaces
- Per-workspace limit: 100k records deleted per day
- Calendar-based retention: records deleted on day X are cleaned up X+14
days later (at midnight UTC boundaries)
### Architecture
- **Cron (WorkspaceTrashCleanupCronJob):** Runs daily, enqueues jobs in
parallel for all workspaces
- **Job (WorkspaceTrashCleanupJob):** Processes individual workspace
cleanup
- **Service (WorkspaceTrashCleanupService):** Discovers tables with
`deletedAt`, deletes old records with quota enforcement
- **Command:** `npx nx run twenty-server:command
cron:workspace:cleanup-trash` to register the cron
### Testing
- Unit tests for service with 100% coverage of public API
- Tested quota enforcement, error handling, and edge cases
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
In this PR:
- refactor the upgrade command / upgrade command runner to keep upgrade
command as light as possible (all wrapping logic should go to upgrade
command runner)
- prevent any upgrade if there is at least one workspace.version <
previsousVersion ==> this leads to corrupted state where only core
migrations are run if the self-hoster is skipping a version
This Pr begins the extensibility journey
- adds a `core.cronTrigger` table
- add a oneToMany relation between core.serverlessFunction and
`core.cronTrigger` (one serverlessFunction can be triggered by multiple
cronTriggers)
- add a job to trigger a serverless function
- adds a cron to trigger serverlessFunction (via the trigger job) based
on the core.cronTrigger.setting.pattern
- adds a command to register the cron
- add the command in `cron-register-all.command.ts`
Improve workflow enqueue cron : instead of relying on the cache to know
how many workflows we can enqueue, query the DB. Then set the cache and
process the not started workflows.
Also adding a second cron that will look for workflows enqueued one hour
ago or more and put these back in the not started status. This will
allow the first cron to start these again.
# What
Fully deprecate old relations because we have one bug tied to it and it
make the codebase complex
# How I've made this PR:
1. remove metadata datasource (we only keep 'core') => this was causing
extra complexity in the refactor + flaky reset
2. merge dev and demo datasets => as I needed to update the tests which
is very painful, I don't want to do it twice
3. remove all code tied to RELATION_METADATA /
relation-metadata.resolver, or anything tied to the old relation system
4. Remove ONE_TO_ONE and MANY_TO_MANY that are not supported
5. fix impacts on the different areas : see functional testing below
# Functional testing
## Functional testing from the front-end:
1. Database Reset ✅
2. Sign In ✅
3. Workspace sign-up ✅
5. Browsing table / kanban / show ✅
6. Assigning a record in a one to many / in a many to one ✅
7. Deleting a record involved in a relation ✅ => broken but not tied to
this PR
8. "Add new" from relation picker ✅ => broken but not tied to this PR
9. Creating a Task / Note, Updating a Task / Note relations, Deleting a
Task / Note (from table, show page, right drawer) ✅ => broken but not
tied to this PR
10. creating a relation from settings (custom / standard x oneToMany /
manyToOne) ✅
11. updating a relation from settings should not be possible ✅
12. deleting a relation from settings (custom / standard x oneToMany /
manyToOne) ✅
13. Make sure timeline activity still work (relation were involved
there), espacially with Task / Note => to be double checked ✅ => Cannot
convert undefined or null to object
14. Workspace deletion / User deletion ✅
15. CSV Import should keep working ✅
16. Permissions: I have tested without permissions V2 as it's still hard
to test v2 work and it's not in prod yet ✅
17. Workflows global test ✅
## From the API:
1. Review open-api documentation (REST) ✅
2. Make sure REST Api are still able to fetch relations ==> won't do, we
have a coupling Get/Update/Create there, this requires refactoring
3. Make sure REST Api is still able to update / remove relation => won't
do same
## Automated tests
1. lint + typescript ✅
2. front unit tests: ✅
3. server unit tests 2 ✅
4. front stories: ✅
5. server integration: ✅
6. chromatic check : expected 0
7. e2e check : expected no more that current failures
## Remove // Todos
1. All are captured by functional tests above, nothing additional to do
## (Un)related regressions
1. Table loading state is not working anymore, we see the empty state
before table content
2. Filtering by Creator Tim Ap return empty results
3. Not possible to add Tasks / Notes / Files from show page
# Result
## New seeds that can be easily extended
<img width="1920" alt="image"
src="https://github.com/user-attachments/assets/d290d130-2a5f-44e6-b419-7e42a89eec4b"
/>
## -5k lines of code
## No more 'metadata' dataSource (we only have 'core)
## No more relationMetadata (I haven't drop the table yet it's not
referenced in the code anymore)
## We are ready to fix the 6 months lag between current API results and
our mocked tests
## No more bug on relation creation / deletion
---------
Co-authored-by: Weiko <corentin@twenty.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
In this PR we are
1. cleaning typeORM service by removing connectToDataSource method
2. using workspaceDataSource instead of mainDataSource when possible,
and replacing raw SQL with workspaceRepository methods to use
Simplifying a lot the upgrade system.
New way to upgrade:
`yarn command:prod upgrade`
New way to write upgrade commands (all wrapping is done for you)
```
override async runOnWorkspace({
index,
total,
workspaceId,
options,
}: RunOnWorkspaceArgs): Promise<void> {}
```
Also cleaning CommandModule imports to make it lighter
Proposal:
- Add a method in ActiveWorkspaceCommand to loop over workspace safely
(add counter, add try / catch, provide datasource with fresh cache,
destroy datasource => as we do always do it)
Also in this PR:
- make sure we clear all dataSources (and not only the one on metadata
version in RAM)
Fixes#9827
Also uncovered a conflict with `@objectType('Relation')` and
`@objectType('relation)`
I don't want to address it in this PR so I will create a followup issue
when we close this but I think there's a confusion between
Relation/RelationMetadata, it's unclear what is what
---------
Co-authored-by: Antoine Moreaux <moreaux.antoine@gmail.com>
- Sync metadata to create workflow entities, since those are not behind
a flag anymore
- Seed workflow views
- Seed workspace favorite for workflow
- Put all steps in upgrade command
In this PR:
- remove old versions upgrade commands
- add a 0.40 upgrade command to loop over all INACTIVE workspaces and
either: update to SUSPENDED (if workspaceSchema exists), update them to
SUSPENDED + deletedAt (if workspaceSchema does not exist anymore)
Note: why updating the deleted one to SUSPENDED? Because I plan to
remove INACTIVE case in the enum in 0.41
Tests made on production like database:
- dry-mode
- singleWorkspaceId
- 3 cases : suspended, deleted+suspended, deleted+suspended+delete all
data
As a follow-up of https://github.com/twentyhq/twenty/pull/9304, we are
here creating a migration to run at the next release, aiming at adding
the new aggregate operation options (CountEmpty, CountNotEmpty, ...,
PercentEmpty, PercentNotEmpty) to the enums on View and ViewField's
aggregateOperations fields.
---------
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
- Added all usable composite field types on pet custom object
- Fixed missing createdBy on people and company seeds
- DEFAULT_SUBDOMAIN is now used for login (could be improved for multi
workspace)
- Refactored ObjectMetadataStandardIdToIdMap to disambiguate from
ObjectMetadataMap
- Refactored seedCustomObjects
Fixes https://github.com/twentyhq/twenty/issues/8810
Fixes https://github.com/twentyhq/twenty/issues/5268
Fixes https://github.com/twentyhq/twenty/issues/8971
- Fixing Task/Note creation not sending position during creation
- Adding a command to backfill position being null, using existing
backfill command.
- Removed unused backfill job.
- Updated workspace entities to set position non-nullable and set a
default value to make it non-required on the API
- Updated position factory to set a default position for all objects
having a POSITION field instead of only company/people
- Moved the try/catch in each resolver factory calling
GraphqlQueryRunnerException handler, makes more sense to call it in the
actual graphql-query-runner and removing some duplicate codes
- Adding validations for input in QueryRunnerArgs factories
- Allow sync-metadata to override and sync defaultValues for certain
field types (that can't be updated by users)
- Removing health-check from sync-metadata command during force mode to
improve performances
## Context
Fix add-subdomain-to-workspace command not included in global module
also fixing the command regex logic that was not generating subdomain
properly
## Context
- Fixing folder structure where 0.33 was inside 0.32.
- Updating the command so it runs enforce uniqueness on all tables
- Updating workspaces entities so the sync metadata adds the index
Search vector fields based on `RICH_TEXT` fields were generated using a
treatment on `RICH_TEXT` fields's `body`
column, to only extract and index the core text.
([PR](https://github.com/twentyhq/twenty/pull/7953))
Actually our RICH_TEXT fields are of datatype `text` in our database,
allowing users to insert non-json values, which breaks the search vector
generation (as it expects json values).
Our vision is unclear for now: for instance we may want to turn
RICH_TEXT into a composite field where the plain text would be stored in
a different column.
So for now, we will (1) treat rich_text data as text, and (2) update the
search vector expressions for the existing workspaces.
## Context
Moving this command to the global command runner for 0.32 upgrade. This
should fix searchVector expression introduced in 0.31 to later handle
soft-deleted + search filter properly.
## Test
```
git co v0.31.0
-- reset DB
git co c--add-simplify-search-vector-expression-to-upgrade-0-32
-- migrate typeorm
-- upgrade 0.32 command
```
search is working and expression is correctly set. Soft deleted records
are not shown as before but it's now possible to override that behavior
via filters.
cc @ijreilly
By default, when custom fields are created, a searchVector field is
created based on the "name" field, which is also the label identifier by
default.
When this label identifier is updated, we want to update the
searchVector field to use this field as searchable field instead, if it
is of "searchable type" (today it is only possible to select a text or
number field as label identifier, while number fields are not
searchable).
Add support for indexes on composite fields and unicity constraint on
indexes
This pull request includes several changes across multiple files to
improve error handling, enforce unique constraints, and update database
migrations. The most important changes include updating error messages
for snack bars, adding a new command to enforce unique constraints, and
updating database migrations to include new fields and constraints.
### Error Handling Improvements:
*
[`packages/twenty-front/src/modules/error-handler/components/PromiseRejectionEffect.tsx`](diffhunk://#diff-e7dc05ced8e4730430f5c7fcd0c75b3aa723da438c26e0bef8130b614427dd9aL23-R23):
Updated error messages in `enqueueSnackBar` to use `error.message`
directly.
*
[`packages/twenty-front/src/modules/object-metadata/hooks/useFindManyObjectMetadataItems.ts`](diffhunk://#diff-74c126d6bc7a5ed6b63be994d298df6669058034bfbc367b11045f9f31a3abe6L44-R46):
Simplified error messages in `enqueueSnackBar`.
*
[`packages/twenty-front/src/modules/object-record/hooks/useFindDuplicateRecords.ts`](diffhunk://#diff-af23a1d99639a66c251f87473e63e2b7bceaa4ee4f70fedfa0fcffe5c7d79181L56-R58):
Simplified error messages in `enqueueSnackBar`.
*
[`packages/twenty-front/src/modules/object-record/hooks/useHandleFindManyRecordsError.ts`](diffhunk://#diff-da04296cbe280202a1eaf6b1244a30490d4f400411bee139651172c59719088eL22-R24):
Simplified error messages in `enqueueSnackBar`.
### New Command for Unique Constraints:
*
[`packages/twenty-server/src/database/commands/upgrade-version/0-31/0-31-enforce-unique-constraints.command.ts`](diffhunk://#diff-8337096c8c80dd2619a5ba691ae5145101f8ae0368a75192a050047e8c6ab7cbR1-R159):
Added a new command to enforce unique constraints on company domain
names and person emails.
*
[`packages/twenty-server/src/database/commands/upgrade-version/0-31/0-31-upgrade-version.command.ts`](diffhunk://#diff-20215e9981a53c7566e9cbff96715685125878f5bcb84fe461a7440f2e68f6fcR13-R14):
Integrated the new `EnforceUniqueConstraintsCommand` into the upgrade
process.
[[1]](diffhunk://#diff-20215e9981a53c7566e9cbff96715685125878f5bcb84fe461a7440f2e68f6fcR13-R14)
[[2]](diffhunk://#diff-20215e9981a53c7566e9cbff96715685125878f5bcb84fe461a7440f2e68f6fcR31)
[[3]](diffhunk://#diff-20215e9981a53c7566e9cbff96715685125878f5bcb84fe461a7440f2e68f6fcR64-R68)
*
[`packages/twenty-server/src/database/commands/upgrade-version/0-31/0-31-upgrade-version.module.ts`](diffhunk://#diff-da52814efc674c25ed55645f8ee2561013641a407f88423e705dd6c77b405527R7):
Registered the new `EnforceUniqueConstraintsCommand` in the module.
[[1]](diffhunk://#diff-da52814efc674c25ed55645f8ee2561013641a407f88423e705dd6c77b405527R7)
[[2]](diffhunk://#diff-da52814efc674c25ed55645f8ee2561013641a407f88423e705dd6c77b405527R24)
### Database Migrations:
*
[`packages/twenty-server/src/database/typeorm/metadata/migrations/1726757368824-migrationDebt.ts`](diffhunk://#diff-c450aeae7bc0ef4416a0ade2dc613ca3f688629f35d2a32f90a09c3f494febdcR1-R53):
Added a migration to update the `relationMetadata_ondeleteaction_enum`
and set default values.
*
[`packages/twenty-server/src/database/typeorm/metadata/migrations/1726757368825-addIsUniqueToIndexMetadata.ts`](diffhunk://#diff-8f1e14bd7f6835ec2c3bb39bcc51e3c318a3008d576a981e682f4c985e746fbfR1-R19):
Added a migration to include the `isUnique` field in `indexMetadata`.
*
[`packages/twenty-server/src/database/typeorm/metadata/migrations/1726762935841-addCompostiveColumnToIndexFieldMetadata.ts`](diffhunk://#diff-7c96b7276c7722d41ff31de23b2de4d6e09adfdc74815356ba63bc96a2669440R1-R19):
Added a migration to include the `compositeColumn` field in
`indexFieldMetadata`.
*
[`packages/twenty-server/src/database/typeorm/metadata/migrations/1726766871572-addWhereToIndexMetadata.ts`](diffhunk://#diff-26651295a975eb50e672dce0e4e274e861f66feb1b68105eee5a04df32796190R1-R14):
Added a migration to include the `indexWhereClause` field in
`indexMetadata`.
### GraphQL Exception Handling:
*
[`packages/twenty-server/src/engine/api/graphql/workspace-query-runner/utils/workspace-query-runner-graphql-api-exception-handler.util.ts`](diffhunk://#diff-58445eb362dc89e31107777d39b592d7842d2ab09a223012ccd055da325270a8R1-R4):
Enhanced exception handling for `QueryFailedError` to provide more
specific error messages for unique constraint violations.
[[1]](diffhunk://#diff-58445eb362dc89e31107777d39b592d7842d2ab09a223012ccd055da325270a8R1-R4)
[[2]](diffhunk://#diff-58445eb362dc89e31107777d39b592d7842d2ab09a223012ccd055da325270a8R23-R59)
*
[`packages/twenty-server/src/engine/api/graphql/workspace-resolver-builder/factories/create-many-resolver.factory.ts`](diffhunk://#diff-233d58ab2333586dd45e46e33d4f07e04a4b8adde4a11a48e25d86985e5a7943L58-R58):
Updated the `workspaceQueryRunnerGraphqlApiExceptionHandler` call to
include context.
*
[`packages/twenty-server/src/engine/api/graphql/workspace-resolver-builder/factories/create-one-resolver.factory.ts`](diffhunk://#diff-68b803f0762c407f5d2d1f5f8d389655a60654a2dd2394a81318655dcd44dc43L58-R58):
Updated the `workspaceQueryRunnerGraphqlApiExceptionHandler` call to
include context.
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
In this PR:
- remove deprecated EMAIL, PHONE, LINK field types (except for Zapier
package as there is another work ongoing)
- remove composite currency filter on currencyCode, actor filter on name
and workspaceMember as the UX is not great yet
This PR was created by [GitStart](https://gitstart.com/) to address the
requirements from this ticket:
[TWNTY-6260](https://clients.gitstart.com/twenty/5449/tickets/TWNTY-6260).
This ticket was imported from:
[TWNTY-6260](https://github.com/twentyhq/twenty/issues/6260)
---
### Description
This is the second PR on TWNTY-6260 which handles data migration of
Phone field to Phones field.\
\
How to Test?\
Follow the below steps:
- On the main branch,
- go to
`packages/twenty-server/src/database/typeorm-seeds/workspace/people.ts`
and change any person's phone number to a string with characters for
example: "test invalid phone", and then reset the DB.
- reset database using `npx nx database:reset twenty-server`
- This is to make sure that invalid numbers will be handled properly. We
should use the invalid value itself to avoid removing data and see how
the behavior is on the front end. should be the same as in the main, the
display shows the invalid value, but the input is empty when you click,
and then you can update.
- Checkout to `TWNTY-6260-phone-migration` branch
- Rebuild typescript using `npx nx build twenty-server`
- Run command `yarn command:prod upgrade-0.32` to do migration
- Run both backend and frontend to see the migrated field
### Demo
- **Loom Video:**\
<https://www.loom.com/share/4b9bcb423cee447d8ad09852a83b27da?sid=ed74ecaa-0339-4575-acdc-a863e95e94fd>
### Refs
#6260
---------
Co-authored-by: gitstart-twenty <gitstart-twenty@users.noreply.github.com>
Co-authored-by: Marie Stoppa <marie.stoppa@essec.edu>
Co-authored-by: Weiko <corentin@twenty.com>
In this PR:
- removing ugprade-0.24 commands as we are releasing 0.30
- introducing cache:flush command
- refactoring upgrade command and sync-metadata command to use the
ActiveWorkspacesCommand so they consistently run on all workspaces or
selected workspaces
Fixes:
- clear localStorage on sign out
- fix missing workspaceMember in verify resolver
- do not throw on datasource already destroyed exception which can
happen with race condition when several resolvers are resolving in
parallel
This is the second PR on TWNTY-6261 which handlesdata migration of Email
field to Emails field.\
\
How to Test?\
Firstly make sure that you have completed the testing steps on first PR
then follow the below steps:
- Checkout to TWNTY-6261-emails-migrations branch
- Rebuild typescript using "npx nx build twenty-server"
- Run command "yarn command:prod upgrade-0.25" to do migration\
\
Loom Video:\
<https://www.loom.com/share/f82b8d29f8f64f92abe3c59c01147b45?sid=9f8ccc05-aa38-4c49-b139-fd0823066273>
**Testing Messaging Sync functionality:**
Please watch the below video to see that the synchronization of contacts
is working fine after migrating Email field to Emails field:\
<https://www.loom.com/share/400949464b244272b78c25e338cc6ab2?sid=103f6625-5933-4b99-9825-0fed33782f36>
**Question to the client**
should we rename email to emails here? in the DomainName PR, the name
did not change.
```typescript
@WorkspaceField({
standardId: PERSON_STANDARD_FIELD_IDS.email,
type: FieldMetadataType.EMAILS,
label: 'Email',
description: 'Contact’s Email',
icon: 'IconMail',
})
email: EmailsMetadata;
```
**Test Messaging Sync**
This pr will update messaging sync files so the changes shouldn't break
existing functionality of importing people and companies in the app.\
To test messaging sync you should follow the below steps:\
1. you need to connect a google account to see the importing
functionality. For this purpose you
have to create a project inside Google Cloud. But to make things easier
you can use the below credentials of an already created project. Put
them in .env of twenty-server package:
```properties
MESSAGING_PROVIDER_GMAIL_ENABLED=true
CALENDAR_PROVIDER_GOOGLE_ENABLED=true
AUTH_GOOGLE_ENABLED=true
AUTH_GOOGLE_CLIENT_ID=951231465939-h61tg6nkpkv1821qi899fjbj9looquto.apps.googleusercontent.com
AUTH_GOOGLE_CLIENT_SECRET=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
AUTH_GOOGLE_CALLBACK_URL=http://localhost:3000/auth/google/redirect
AUTH_GOOGLE_APIS_CALLBACK_URL=http://localhost:3000/auth/google-apis/get-access-token
MESSAGE_QUEUE_TYPE=bull-mq
```
Alternative env
```properties
MESSAGING_PROVIDER_GMAIL_ENABLED=true
CALENDAR_PROVIDER_GOOGLE_ENABLED=true
AUTH_GOOGLE_ENABLED=true
AUTH_GOOGLE_CLIENT_ID=622006708006-dc4n3vrtf3cs2h6k7hgbborudme7ku9l.apps.googleusercontent.com
AUTH_GOOGLE_CLIENT_SECRET=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
AUTH_GOOGLE_CALLBACK_URL=http://localhost:3000/auth/google/redirect
AUTH_GOOGLE_APIS_CALLBACK_URL=http://localhost:3000/auth/google-apis/get-access-token
MESSAGE_QUEUE_TYPE=bull-mq
```
1. Launch your worker with `npx nx run twenty-server:worker`
2. npx nx run twenty-server:command cron:messaging:messages-import
3. npx nx run twenty-server:command cron:messaging:message-list-fetch
4. npx nx run twenty-server:command
cron📆calendar-event-list-fetch
5. Run the app and navigate to Settings/Accounts then connect your
Google account
---------
Co-authored-by: gitstart-twenty <gitstart-twenty@users.noreply.github.com>
Co-authored-by: Marie Stoppa <marie.stoppa@essec.edu>
Co-authored-by: Weiko <corentin@twenty.com>
## Context
As we grow, the messaging scripts are experiencing performance issues
forcing us to temporarily disable them on the cloud.
While investigating the performance, I have noticed that generating the
entity schema (for twentyORM) in the repository is taking ~500ms locally
on my Mac M2 so likely more on pods. Caching the entitySchema then!
I'm also clarifying naming around schemaVersion and cacheVersions ==>
both are renamed workspaceMetadataVersion and migrated to the workspace
table (the workspaceCacheVersion table is dropped).
