# Introduction
Typeorm migration are now associated to a given twenty-version from the
`UPGRADE_COMMAND_SUPPORTED_VERSIONS` that the current twenty core engine
handles
This way when we upgrade we retrieve the migrations that need to be run,
this will be useful for the cross-version incremental upgrade so we
preserve sequentiality
## What's new
To generate
```sh
npx nx database:migrate:generate twenty-server -- --name add-index-to-users
```
To apply all
```sh
npx nx database:migrate twenty-server
```
## Next
Introduce slow and fast typeorm migration in order to get rid of the
save point pattern in our code base
Create a clean and dedicated `InstanceUpgradeService` abstraction
## Motivations
A lot of self hosters hands up using the `yarn database:migrated:prod`
either manually or through AI assisted debug while they try to upgrade
an instance while their workspace is still blocked in a previous one
Leading to their whole database permanent corruption
## What happened
Replaced the direct call the the typeorm cli to a command calling it
programmatically, adding a layer of security in case a workspace seems
to be blocked in a previous version than the one just before the one
being installed ( e.g 1.0 when you try to upgrade from 1.1 to 1.2 )
For our cloud we still need a way to bypass this security explaining the
-f flag
## Remark
Centralized this logic and refactored creating new services
`WorkspaceVersionService` and `CoreEngineVersionService` that will
become useful for the upcoming upgrade refactor
Related to https://github.com/twentyhq/twenty-infra/pull/529
## Summary
- The `ci-ai-catalog-sync` cron workflow was failing because the
`ai:sync-models-dev` NestJS command bootstraps the full app, which tries
to connect to PostgreSQL — unavailable in CI.
- Converted the sync logic to a standalone `ts-node` script
(`scripts/ai-sync-models-dev.ts`) that runs without NestJS, eliminating
the database dependency.
- Removed the `Build twenty-server` step from the workflow since it's no
longer needed, making the job faster.
## Test plan
- [x] Verified the standalone script runs successfully locally via `npx
nx run twenty-server:ts-node-no-deps-transpile-only --
./scripts/ai-sync-models-dev.ts`
- [x] Verified `--dry-run` flag works correctly
- [x] Verified the output `ai-providers.json` is correctly written with
valid JSON (135 models across 5 providers)
- [x] Verified the script passes linting with zero errors
- [ ] CI should pass without requiring a database service
Fixes:
https://github.com/twentyhq/twenty/actions/runs/23424202182/job/68135439740
Made with [Cursor](https://cursor.com)
## Summary
- Replaces per-provider TypeScript constant files
(`openai-models.const.ts`, `anthropic-models.const.ts`, etc.) with a
single `ai-providers.json` catalog as the source of truth
- Adds runtime model discovery via AI SDK for self-hosted providers,
with `models.dev` enrichment for pricing/capabilities
- Introduces composite model IDs (`provider/modelId`) for canonical,
conflict-free identification
- Simplifies provider configuration: API keys are injected from
environment variables (e.g., `OPENAI_API_KEY`)
- Adds admin panel UI for provider management (add/remove/test), model
discovery, recommended model configuration, and default fast/smart model
selection per workspace
- Removes deprecated config variables (`AI_DISABLED_MODEL_IDS`,
`AUTO_ENABLE_NEW_AI_MODELS`, etc.)
- Adds database migration for composite model ID format
## Test plan
- [ ] Server typecheck passes
- [ ] Frontend typecheck passes
- [ ] Server unit tests pass
- [ ] Frontend unit tests pass
- [ ] CI pipeline green
- [ ] Admin panel AI tab loads correctly
- [ ] Provider discovery works for configured providers
- [ ] Model recommendation toggles persist
- [ ] Default fast/smart model selection works
Made with [Cursor](https://cursor.com)
This PR adds a `workspace:export` server command for ongoing debug
tooling/administrative work
Demo Video shows
1. Exporting a sample workspace YC
2. Restoring Local DB Docker volume snapshot to Dropped YC Workspace
3. Importing exported SQL
4. Booting and navigating the imported Workspace
https://github.com/user-attachments/assets/0e1ac6cb-8ce1-440b-8b56-f81dcb27a9c8
## Summary
This PR implements OAuth 2.0 Dynamic Client Registration (RFC 7591) and
OAuth 2.0 Protected Resource Metadata (RFC 9728) support, enabling
third-party applications to dynamically register as OAuth clients
without manual configuration.
## Key Changes
### OAuth Dynamic Client Registration
- **New Controller**: `OAuthRegistrationController` at `POST
/oauth/register` endpoint
- Validates client metadata according to RFC 7591 specifications
- Enforces PKCE-only public client model (no client secrets)
- Supports only `authorization_code` grant type and `code` response type
- Rate limits registrations to 10 per hour per IP address
- Returns `client_id` and registration metadata in response
- **Input Validation**: `OAuthRegisterInput` DTO with constraints on:
- Client name (max 256 chars)
- Redirect URIs (max 20, validated for security)
- Grant types, response types, scopes, and auth methods
- Logo and client URIs (max 2048 chars)
- **Discovery Endpoint Update**: Added `registration_endpoint` to OAuth
discovery metadata
### Stale Registration Cleanup
- **Cleanup Service**: Automatically removes OAuth-only registrations
older than 30 days that have no active installations
- **Cron Job**: Runs daily at 02:30 AM UTC with batch processing (100
records per batch)
- **CLI Command**: `cron:stale-registration-cleanup` to manually trigger
cleanup
### MCP (Model Context Protocol) Authentication
- **New Guard**: `McpAuthGuard` implements RFC 9728 compliance
- Wraps JWT authentication with proper error responses
- Returns `WWW-Authenticate` header with protected resource metadata URL
on 401
- Enables OAuth-protected MCP endpoints
### Protected Resource Metadata
- **New Endpoint**: `GET /.well-known/oauth-protected-resource` (RFC
9728)
- Advertises MCP resource as OAuth-protected
- Lists supported scopes and bearer token methods
- Enables OAuth clients to discover authorization requirements
### Application Registration Updates
- **New Source Type**: `OAUTH_ONLY` enum value for OAuth-only
registrations
- **Install Service**: Skips artifact installation for OAuth-only apps
(no code artifacts)
### Frontend Updates
- **Authorization Page**: Support both snake_case (standard OAuth) and
camelCase (legacy) query parameters
- `client_id` / `clientId`
- `code_challenge` / `codeChallenge`
- `redirect_uri` / `redirectUrl`
## Implementation Details
- **Rate Limiting**: Uses token bucket algorithm with 10 registrations
per 3,600,000ms window per IP
- **Scope Validation**: Requested scopes are capped to allowed OAuth
scopes; defaults to all scopes if not specified
- **Redirect URI Validation**: Uses existing `validateRedirectUri`
utility for security
- **Cache Headers**: Registration responses include `Cache-Control:
no-store` and `Pragma: no-cache`
- **Batch Processing**: Cleanup operations process 100 records at a time
to avoid memory issues
- **Grace Period**: 30-day grace period before cleanup to allow time for
client activation
https://claude.ai/code/session_01PxcuWFFRuXMASMaMGTLYk2
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: github-actions <github-actions@twenty.com>
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
## Summary
Implements enterprise licensing and per-seat billing for self-hosted
environments, with Stripe as the single source of truth for subscription
data.
### Components
- **twenty-website** hosts the private key to sign `ENTERPRISE_KEY` and
`ENTERPRISE_VALIDITY_TOKEN`. It communicates with Stripe to emit the
daily `ENTERPRISE_VALIDITY_TOKEN` if the subscription is active, based
on the user's Stripe subscription ID stored in `ENTERPRISE_KEY`.
- **Stripe** is the single source of truth for subscription data
(status, seats, billing).
- **The client** (twenty-server + DB + workers) saves `ENTERPRISE_KEY`
in the `keyValuePair` table (or `.env` if
`IS_CONFIG_VARIABLES_IN_DB_ENABLED` is false) and the daily-renewed
`ENTERPRISE_VALIDITY_TOKEN` in the `appToken` table.
`ENTERPRISE_VALIDITY_TOKEN` is verified client-side using a public key
to grant access to enterprise features (RLS, SSO, audit logs, etc.).
### Flow
1. When requesting an upgrade to an enterprise plan (from **Enterprise**
in settings), the user is shown a modal to choose monthly/yearly
billing, then redirected to Stripe to enter payment details. After
checkout, they land on twenty-website where they are exposed to their
`ENTERPRISE_KEY`, which they paste in the UI. It is saved in the
`keyValuePair` table. On activation, a first `ENTERPRISE_VALIDITY_TOKEN`
with 30-day validity is stored in the `appToken` table.
2. **Every day**, a cron job runs and does two things:
- **Refreshes the validity token**: communicates with twenty-website to
get a new `ENTERPRISE_VALIDITY_TOKEN` with 30-day validity if the Stripe
subscription is still active. If the subscription is in cancellation,
the emitted token has a validity equal to the cancellation date. If it's
no longer valid, the token is not replaced. The cron only needs to run
every 30 days in practice, but runs daily so it's resilient to
occasional failures.
- **Reports seat count**: counts active (non-soft-deleted)
`UserWorkspace` entries and sends the count to twenty-website, which
updates the Stripe subscription quantity with proration. Seats are also
reported on first activation. If the subscription is canceled or
scheduled for cancellation, the seat update is skipped.
3. `ENTERPRISE_VALIDITY_TOKEN` is verified server-side via a public key
to grant access to enterprise features.
### Key concepts
Three distinct checks are exposed as GraphQL fields on `Workspace`:
| Field | Meaning |
|---|---|
| `hasValidEnterpriseKey` | Has any valid enterprise key (signed JWT
**or** legacy plain string) |
| `hasValidSignedEnterpriseKey` | `ENTERPRISE_KEY` is a properly signed
JWT (billing portal makes sense) |
| `hasValidEnterpriseValidityToken` | `ENTERPRISE_VALIDITY_TOKEN` is
present and not expired (expiration depends on signed token payload, not
on "expiresAt" on appToken table which is only indicative) |
Feature access is gated by `isValid()` =
`hasValidEnterpriseValidityToken || hasValidEnterpriseKey` (to support
both new and legacy keys during transition). After transition isValid()
= hasValidEnterpriseValidityToken
### Frontend states
The Enterprise settings page handles multiple states:
- **No key**: show "Get Enterprise" with checkout modal
- **Orphaned validity token** (token valid but no signed key): prompt
user to set a valid enterprise key
- **Active/trialing but no validity token**: show subscription status
with a "Reload validity token" action
- **Active/trialing**: show full subscription info, billing portal
access, cancel option
- **Cancellation scheduled**: show cancellation date, billing portal
- **Canceled**: show billing history link and option to start a new
subscription
- **Past due / Incomplete**: prompt to update payment or restart
### Temporary retro-compatibility: legacy plain-text keys
Previously, enterprise features were gated by a simple check: any
non-empty string in `ENTERPRISE_KEY` granted access. With this PR, we
transition to a controlled system relying on signed JWTs.
To avoid breaking existing self-hosted users:
- **Legacy plain-text keys still grant access** to enterprise features.
`hasValidEnterpriseKey` returns `true` for both signed JWTs and plain
strings, and `isValid()` checks `hasValidEnterpriseKey` as a fallback
when no validity token is present.
- **A deprecation banner** is shown at the top of the app when
`hasValidEnterpriseKey` is `true` but `hasValidSignedEnterpriseKey` is
`false`, informing the user that their key format is deprecated and they
should activate a new signed key.
- **No billing portal or subscription management** is available for
legacy keys since there is no Stripe subscription to manage.
This retro-compatibility will be removed in a future version. At that
point, `isValid()` will only check `hasValidEnterpriseValidityToken`.
### Edge cases
- **Air-gapped / production environments**: for self-hosted clients that
block external traffic (or for our own production), provide a long-lived
`ENTERPRISE_VALIDITY_TOKEN` (e.g. 99 years) directly in the `appToken`
table, with no `ENTERPRISE_KEY`. The daily cron will skip the refresh
(no enterprise key to authenticate with), but the pre-seeded validity
token will be used to grant feature access. No billing or seat reporting
occurs in this mode.
- **`IS_CONFIG_VARIABLES_IN_DB_ENABLED` is false**: if the user tries to
activate an enterprise key but DB config writes are disabled, the
backend returns a clear error asking them to add `ENTERPRISE_KEY` to
their `.env` file manually.
- **Canceled subscriptions**: the `/seats` endpoint skips Stripe updates
for canceled or cancellation-scheduled subscriptions to avoid Stripe API
errors.
### How to test
- launch twenty-website on a different url (eg localhost:1002)
- add ENTERPRISE_API_URL=http://localhost:3002/api/enterprise (or else)
in your server .env
- ask me for twenty-website's .env file content (STRIPE_SECRET_KEY;
STRIPE_ENTERPRISE_MONTHLY_PRICE_ID;STRIPE_ENTERPRISE_YEARLY_PRICE_ID;
ENTERPRISE_JWT_PRIVATE_KEY; ENTERPRISE_JWT_PUBLIC_KEY;
NEXT_PUBLIC_WEBSITE_URL)
- visit Admin panel / enterprise
## Summary
- **Module reorganization**: Moved `ApplicationUpgradeService` and cron
jobs to `application-upgrade/`, `ApplicationSyncService` to
`application-manifest/`, and
`runWorkspaceMigration`/`uninstallApplication` mutations to the manifest
resolver — each module now has a single clear responsibility.
- **Explicit install flow**: Removed implicit `ApplicationEntity`
creation from `ApplicationSyncService`. The install service and dev
resolver now explicitly create the `ApplicationEntity` before syncing.
npm packages are resolved at registration time to extract manifest
metadata (universalIdentifier, name, description, etc.), eliminating the
`reconcileUniversalIdentifier` hack.
- **Better error handling**: Frontend hooks now surface actual server
error messages in snackbars instead of swallowing them. Replaced the
ugly `ConfirmationModal` for transfer ownership with a proper form
modal. Fixed `SettingsAdminTableCard` row height overflow and corrected
the `yarn-engine` asset path.
## Test plan
- [ ] Register an npm package — verify manifest metadata (name,
description, universalIdentifier) is extracted correctly
- [ ] Install a registered npm app on a workspace — verify
ApplicationEntity is created and sync succeeds
- [ ] Test `app:dev` CLI flow — verify local app registration and sync
work
- [ ] Upload a tarball — verify registration and install flow
- [ ] Transfer ownership — verify the new modal UX works
- [ ] Verify error messages appear correctly in snackbars when
operations fail
Made with [Cursor](https://cursor.com)
# Introduction
Important note: This PR officially deprecates the `standardId`, about to
drop col and entity property after this has been merged
Important note2: Haven't updated the optimistic tool to also update the
universal identifier aggregators only the ids one, they should not be
consumed in the runner context -> need to improve typing or either the
optimistic tooling
In this PR we're introducing all the devxp allowing future metadata
incremental universal migration -> this has an impact on all existing
metadata actions handler ( explaining its size )
This PR also introduce workspace agnostic create update actions runner
for both field and object metadata in order to battle test the described
above devxp
Noting that these two metadata are the most complex to handle
Notes:
- A workspace migration is now highly bind to a
`applicationUniversalIdentifier`. Though we don't strictly validate
application scope for the moment
## Next
Migrate both object and field builder to universal comparison
## Universal Actions vs Flat Actions Architecture
### Concept
The migration system uses a two-phase action model:
1. **Universal Actions** - Actions defined using `universalIdentifier`
(stable, portable identifiers like `standardId` + `applicationId`)
2. **Flat Actions** - Actions defined using database `entityId` (UUIDs
specific to a workspace)
### Why This Separation?
- **Universal actions are portable**: They can be serialized, stored,
and replayed across different workspaces
- **Flat actions are executable**: They contain the actual database IDs
needed to perform operations
- **Decoupling**: The builder produces universal actions; the runner
transpiles them to flat actions at execution time
### Transpiler Pattern
Each action handler must implement
`transpileUniversalActionToFlatAction()`:
```typescript
@Injectable()
export class CreateFieldActionHandlerService extends WorkspaceMigrationRunnerActionHandler(
'create',
'fieldMetadata',
) {
override async transpileUniversalActionToFlatAction(
context: WorkspaceMigrationActionRunnerArgs<UniversalCreateFieldAction>,
): Promise<FlatCreateFieldAction> {
// Resolve universal identifiers to database IDs
const flatObjectMetadata = findFlatEntityByUniversalIdentifierOrThrow({
flatEntityMaps: allFlatEntityMaps.flatObjectMetadataMaps,
universalIdentifier: action.objectMetadataUniversalIdentifier,
});
return {
type: action.type,
metadataName: action.metadataName,
objectMetadataId: flatObjectMetadata.id, // Resolved ID
flatFieldMetadatas: /* ... transpiled entities ... */,
};
}
}
```
### Action Handler Base Class
`BaseWorkspaceMigrationRunnerActionHandlerService<TActionType,
TMetadataName>` provides:
- **`transpileUniversalActionToFlatAction()`** - Abstract method each
handler must implement
- **`transpileUniversalDeleteActionToFlatDeleteAction()`** - Shared
helper for delete actions
## FlatEntityMaps custom properties
Introduced a `TWithCustomMapsProperties` generic parameter to control
whether custom indexing structures are included:
- **`false` (default)**: Returns `FlatEntityMaps<MetadataFlatEntity<T>>`
- used in builder/runner contexts
- **`true`**: Returns the full maps type with custom properties (e.g.,
`byUserWorkspaceIdAndFolderId`) - used in cache contexts
## Create Field Actions Refactor
Refactored create-field actions to support relation field pairs
bundling.
**Problem:** Relation fields (e.g., `Attachment.targetTask` ↔
`Task.attachments`) couldn't resolve each other's IDs during
transpilation because they were in separate actions with independent
`fieldIdByUniversalIdentifier` maps.
**Solution:**
- Removed `objectMetadataUniversalIdentifier` from
`UniversalCreateFieldAction` and `objectMetadataId` from
`FlatCreateFieldAction` - each field now carries its own
- Runner groups fields by object internally and processes each table
separately
- Split aggregator into two focused utilities:
- `aggregateNonRelationFieldsIntoObjectActions` - merges non-relation
fields into object actions
- `aggregateRelationFieldPairs` - bundles relation pairs with shared
`fieldIdByUniversalIdentifier`
# Introduction
Followup of
https://github.com/twentyhq/twenty/pull/17001#pullrequestreview-3638508738
close https://github.com/twentyhq/core-team-issues/issues/1910
We've completely decom the `sync-metadata` in production. We're now then
removing its implementation in favor of the v2.
## TODO:
- [x] Remove sync-metadata implem and commands
- [x] Remove workspace decorators
- [x] Type each deprecated field to deprecated on their workspaceEntity
- [x] Remove the `workspace-sync-metadata` folder entirely
- [x] remove workspace migration
- [x] workspace migration removal migration
- [x] remove the `v2` references from workspace manager file names
- [x] remove the `v2` references from workspace manager modules
- [ ] Double check impact on translation file path updates
## Note
- Removed the gate logic
- Remains some service v2 naming, serverless needs to be migrated on v2
fully
- Removed workspaceMigration service app health consumption, making it
always returning up ( no more down ) cc @FelixMalfait ( quite obsolete
health check now, will require complete refactor once we introduce inter
app dependency etc )
# Introduction
Followup https://github.com/twentyhq/twenty/pull/16863
Important note: This is not an upgrade command and will have to be
manually run
In this pull request we're introducing a coding that will allow this
[migration](https://github.com/twentyhq/twenty/blob/clean-orphan-metadata/packages/twenty-server/src/database/typeorm/core/migrations/utils/1767002571103-addWorkspaceForeignKeys.util.ts#L3)
to pass, it enforces the `workspaceId` foreignKey on all metadata
entities. Allowing workspace deletion cascading of all its related
entities and avoiding orphan metadata entities to reoccur in the future
Also introduced a small migration that will set the workspaceId col type
to `uuid`, as it has been historically `varchar`
This migration is a requirement for the above command to work
successfully
## Note
Chunking by relations fields the orphan field deletion as would take way
too much time within a transac,
## Test
Tested on a prod extract locally ( both dry and not dry )
# Introduction
Related to https://github.com/twentyhq/core-team-issues/issues/1995
This PR introduces the basis of the `twentyStandard` application as code
on demand, it's highly tied to `ids` where it will becomes workspace
agnostic following the builder and runner `universalIdentifier` refactor
later.
The goal here to allow computing the `allFlatEntityMaps` `to` of the
`twentyStandard` application on a empty workspace ( workspace creation
). Allowing installing the twenty standard app through a workspace
migration instead of passing by the sync metadata
Nothing done will be run in production for the moment if it's not the
small validation refactor we've introduced
Please note that everything introduced here will be replaced at some
point by a twenty app instance when the twenty sdk is mature enough to
handle of the edge cases we need here
## How we've proceeded
We've been iterating over every workspace entity both objects and their
fields, and transpiled them to flatEntity.
Being sure we migrate the defaultValue, settings and so on accordingly.
We've also compute all the ids in prior of the whole entities
computation so we don't face any hoisting issue.
## Current state
At the moment only handling all of the 29 standard objects and their
fields
Settings a unique universalIdentifier for all of them
Will come views, agent role targets and so on later
## `workspace:compute-twenty-standard-migration` command
This command allow generating a workspace migration that will result in
installing the twenty standard app in an empty workspace
It's temporary and aims to allow debugging for the moment we might not
keep it in the future as it is right now
It contains debug writeFileSync which is expected no worries greptile
## `LabelFieldMetadataIdentifierId`
Small refactor allowing defining the label identifier field metadata id
of a uuid field metadata type for system object, as some of our standard
object don't have a name field and don't aim to
Also please note that we might remove this build options later in the
sake of the currently installed universal identifier application that we
could compare with the deterministic twenty standard one
## `runFlatFieldMetadataValidators`
Deprecated this pattern which was redundant and not v2 friendly pattern
## Current errors that will address in upcoming PR
Current standard objects and fields metadata does not pass the
validation that we have in place, as historically the sync metadata
would directly consume the repositories and would just ignore the
validation. This is about to change.
Will handle the below errors in dedicated PRs as they will required
upgrade commands in order to migrate the data, or will handle that from
the sync metadata instead still to be determined but nothing critical
here
- camel case field metadata name
- options label invalid format
```json
{
"status": "fail",
"report": {
"fieldMetadata": [
{
"status": "fail",
"errors": [
{
"code": "INVALID_FIELD_INPUT",
"message": "Name should be in camelCase",
"userFriendlyMessage": {
"id": "P+jdmX",
"message": "Name should be in camelCase"
},
"value": "iCalUID"
}
],
"flatEntityMinimalInformation": {
"id": "68dd83cd-92c8-4233-bb28-47939bab6124",
"name": "iCalUID",
"objectMetadataId": "11c16ab6-9176-439e-a2db-a12c5a58a524"
},
"type": "create_field"
},
{
"status": "fail",
"errors": [
{
"code": "INVALID_FIELD_INPUT",
"message": "Value must be in UPPER_CASE and follow snake_case \"email\"",
"userFriendlyMessage": {
"id": "UBPzFQ",
"message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
"values": {
"sanitizedValue": "email"
}
},
"value": "email"
},
{
"code": "INVALID_FIELD_INPUT",
"message": "Value must be in UPPER_CASE and follow snake_case \"sms\"",
"userFriendlyMessage": {
"id": "UBPzFQ",
"message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
"values": {
"sanitizedValue": "sms"
}
},
"value": "sms"
}
],
"flatEntityMinimalInformation": {
"id": "e3caaf2a-e07d-4146-8dfc-9eef904e82c9",
"name": "type",
"objectMetadataId": "4b777de5-4c7b-4af4-9b92-655c0f87512b"
},
"type": "create_field"
},
{
"status": "fail",
"errors": [
{
"code": "INVALID_FIELD_INPUT",
"message": "Value must be in UPPER_CASE and follow snake_case \"incoming\"",
"userFriendlyMessage": {
"id": "UBPzFQ",
"message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
"values": {
"sanitizedValue": "incoming"
}
},
"value": "incoming"
},
{
"code": "INVALID_FIELD_INPUT",
"message": "Value must be in UPPER_CASE and follow snake_case \"outgoing\"",
"userFriendlyMessage": {
"id": "UBPzFQ",
"message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
"values": {
"sanitizedValue": "outgoing"
}
},
"value": "outgoing"
}
],
"flatEntityMinimalInformation": {
"id": "d96233a4-93be-45ea-9548-3b50f3c700cf",
"name": "direction",
"objectMetadataId": "480a648a-d2e5-482a-992f-ef053e1b4bb0"
},
"type": "create_field"
},
{
"status": "fail",
"errors": [
{
"code": "INVALID_FIELD_INPUT",
"message": "Value must be in UPPER_CASE and follow snake_case \"from\"",
"userFriendlyMessage": {
"id": "UBPzFQ",
"message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
"values": {
"sanitizedValue": "from"
}
},
"value": "from"
},
{
"code": "INVALID_FIELD_INPUT",
"message": "Value must be in UPPER_CASE and follow snake_case \"to\"",
"userFriendlyMessage": {
"id": "UBPzFQ",
"message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
"values": {
"sanitizedValue": "to"
}
},
"value": "to"
},
{
"code": "INVALID_FIELD_INPUT",
"message": "Value must be in UPPER_CASE and follow snake_case \"cc\"",
"userFriendlyMessage": {
"id": "UBPzFQ",
"message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
"values": {
"sanitizedValue": "cc"
}
},
"value": "cc"
},
{
"code": "INVALID_FIELD_INPUT",
"message": "Value must be in UPPER_CASE and follow snake_case \"bcc\"",
"userFriendlyMessage": {
"id": "UBPzFQ",
"message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
"values": {
"sanitizedValue": "bcc"
}
},
"value": "bcc"
}
],
"flatEntityMinimalInformation": {
"id": "961c598e-67c3-452d-8bb2-b92c0bc64404",
"name": "role",
"objectMetadataId": "8af8a13c-ff97-4cd3-b70d-52a7dc2924b4"
},
"type": "create_field"
},
{
"status": "fail",
"errors": [
{
"code": "INVALID_FIELD_INPUT",
"message": "Label must not contain a comma",
"userFriendlyMessage": {
"id": "k731jp",
"message": "Label must not contain a comma"
},
"value": "Commas and dot (1,234.56)"
},
{
"code": "INVALID_FIELD_INPUT",
"message": "Label must not contain a comma",
"userFriendlyMessage": {
"id": "k731jp",
"message": "Label must not contain a comma"
},
"value": "Spaces and comma (1 234,56)"
},
{
"code": "INVALID_FIELD_INPUT",
"message": "Label must not contain a comma",
"userFriendlyMessage": {
"id": "k731jp",
"message": "Label must not contain a comma"
},
"value": "Dots and comma (1.234,56)"
}
],
"flatEntityMinimalInformation": {
"id": "7fa20caf-2597-42e3-84e5-15a91b125b9b",
"name": "numberFormat",
"objectMetadataId": "a6974302-9e72-461c-aa09-9390f4ff16fc"
},
"type": "create_field"
}
],
"objectMetadata": [],
"view": [],
"viewField": [],
"viewGroup": [],
"index": [],
"serverlessFunction": [],
"cronTrigger": [],
"databaseEventTrigger": [],
"routeTrigger": [],
"viewFilter": [],
"role": [],
"roleTarget": [],
"agent": []
}
}
```
**Before**
- any user with workpace_members permission was able to remove a user
from their workspace. This triggered the deletion of workspaceMember +
of userWorkspace, but did not delete the user (even if they had no
workspace left) nor the roleTarget (acts as junction between role and
userWorkspace) which was left with a userWorkspaceId pointing to
nothing. This is because roleTarget points to userWorkspaceId but the
foreign key constraint was not implemented
- any user could delete their own account. This triggered the deletion
of all their workspaceMembers, but not of their userWorkspace nor their
user nor the roleTarget --> we have orphaned userWorkspace, not
technically but product wise - a userWorkspace without a workspaceMember
does not make sense
So the problems are
- we have some roleTargets pointing to non-existing userWorkspaceId
(which caused https://github.com/twentyhq/twenty/issues/14608 )
- we have userWorkspaces that should not exist and that have no
workspaceMember counterpart
- it is not possible for a user to leave a workspace by themselves, they
can only leave all workspaces at once, except if they are being removed
from the workspace by another user
**Now**
- if a user has multiple workspaces, they are given the possibility to
leave one workspace while remaining in the others (we show two buttons:
Leave workspace and Delete account buttons). if a user has just one
workspace, they only see Delete account
- when a user leaves a workspace, we delete their workspaceMember,
userWorkspace and roleTarget. If they don't belong to any other
workspace we also soft-delete their user
- soft-deleted users get hard deleted after 30 days thanks to a cron
- we have two commands to clean the orphans roleTarget and userWorkspace
(TODO: query db to see how many must be run)
**Next**
- once the commands have been run, we can implement and introduce the
foreign key constraint on roleTarget
Fixes https://github.com/twentyhq/twenty/issues/14608
Closes#14726
### Added
- `trashRetentionDays` field to workspace entity (default: 14 days)
- Automated trash cleanup using BullMQ jobs
- Daily cron (00:10 UTC) that enqueues cleanup jobs for all active
workspaces
- Per-workspace limit: 100k records deleted per day
- Calendar-based retention: records deleted on day X are cleaned up X+14
days later (at midnight UTC boundaries)
### Architecture
- **Cron (WorkspaceTrashCleanupCronJob):** Runs daily, enqueues jobs in
parallel for all workspaces
- **Job (WorkspaceTrashCleanupJob):** Processes individual workspace
cleanup
- **Service (WorkspaceTrashCleanupService):** Discovers tables with
`deletedAt`, deletes old records with quota enforcement
- **Command:** `npx nx run twenty-server:command
cron:workspace:cleanup-trash` to register the cron
### Testing
- Unit tests for service with 100% coverage of public API
- Tested quota enforcement, error handling, and edge cases
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
In this PR:
- refactor the upgrade command / upgrade command runner to keep upgrade
command as light as possible (all wrapping logic should go to upgrade
command runner)
- prevent any upgrade if there is at least one workspace.version <
previsousVersion ==> this leads to corrupted state where only core
migrations are run if the self-hoster is skipping a version
This Pr begins the extensibility journey
- adds a `core.cronTrigger` table
- add a oneToMany relation between core.serverlessFunction and
`core.cronTrigger` (one serverlessFunction can be triggered by multiple
cronTriggers)
- add a job to trigger a serverless function
- adds a cron to trigger serverlessFunction (via the trigger job) based
on the core.cronTrigger.setting.pattern
- adds a command to register the cron
- add the command in `cron-register-all.command.ts`
Improve workflow enqueue cron : instead of relying on the cache to know
how many workflows we can enqueue, query the DB. Then set the cache and
process the not started workflows.
Also adding a second cron that will look for workflows enqueued one hour
ago or more and put these back in the not started status. This will
allow the first cron to start these again.
# What
Fully deprecate old relations because we have one bug tied to it and it
make the codebase complex
# How I've made this PR:
1. remove metadata datasource (we only keep 'core') => this was causing
extra complexity in the refactor + flaky reset
2. merge dev and demo datasets => as I needed to update the tests which
is very painful, I don't want to do it twice
3. remove all code tied to RELATION_METADATA /
relation-metadata.resolver, or anything tied to the old relation system
4. Remove ONE_TO_ONE and MANY_TO_MANY that are not supported
5. fix impacts on the different areas : see functional testing below
# Functional testing
## Functional testing from the front-end:
1. Database Reset ✅
2. Sign In ✅
3. Workspace sign-up ✅
5. Browsing table / kanban / show ✅
6. Assigning a record in a one to many / in a many to one ✅
7. Deleting a record involved in a relation ✅ => broken but not tied to
this PR
8. "Add new" from relation picker ✅ => broken but not tied to this PR
9. Creating a Task / Note, Updating a Task / Note relations, Deleting a
Task / Note (from table, show page, right drawer) ✅ => broken but not
tied to this PR
10. creating a relation from settings (custom / standard x oneToMany /
manyToOne) ✅
11. updating a relation from settings should not be possible ✅
12. deleting a relation from settings (custom / standard x oneToMany /
manyToOne) ✅
13. Make sure timeline activity still work (relation were involved
there), espacially with Task / Note => to be double checked ✅ => Cannot
convert undefined or null to object
14. Workspace deletion / User deletion ✅
15. CSV Import should keep working ✅
16. Permissions: I have tested without permissions V2 as it's still hard
to test v2 work and it's not in prod yet ✅
17. Workflows global test ✅
## From the API:
1. Review open-api documentation (REST) ✅
2. Make sure REST Api are still able to fetch relations ==> won't do, we
have a coupling Get/Update/Create there, this requires refactoring
3. Make sure REST Api is still able to update / remove relation => won't
do same
## Automated tests
1. lint + typescript ✅
2. front unit tests: ✅
3. server unit tests 2 ✅
4. front stories: ✅
5. server integration: ✅
6. chromatic check : expected 0
7. e2e check : expected no more that current failures
## Remove // Todos
1. All are captured by functional tests above, nothing additional to do
## (Un)related regressions
1. Table loading state is not working anymore, we see the empty state
before table content
2. Filtering by Creator Tim Ap return empty results
3. Not possible to add Tasks / Notes / Files from show page
# Result
## New seeds that can be easily extended
<img width="1920" alt="image"
src="https://github.com/user-attachments/assets/d290d130-2a5f-44e6-b419-7e42a89eec4b"
/>
## -5k lines of code
## No more 'metadata' dataSource (we only have 'core)
## No more relationMetadata (I haven't drop the table yet it's not
referenced in the code anymore)
## We are ready to fix the 6 months lag between current API results and
our mocked tests
## No more bug on relation creation / deletion
---------
Co-authored-by: Weiko <corentin@twenty.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
In this PR we are
1. cleaning typeORM service by removing connectToDataSource method
2. using workspaceDataSource instead of mainDataSource when possible,
and replacing raw SQL with workspaceRepository methods to use
Simplifying a lot the upgrade system.
New way to upgrade:
`yarn command:prod upgrade`
New way to write upgrade commands (all wrapping is done for you)
```
override async runOnWorkspace({
index,
total,
workspaceId,
options,
}: RunOnWorkspaceArgs): Promise<void> {}
```
Also cleaning CommandModule imports to make it lighter
Proposal:
- Add a method in ActiveWorkspaceCommand to loop over workspace safely
(add counter, add try / catch, provide datasource with fresh cache,
destroy datasource => as we do always do it)
Also in this PR:
- make sure we clear all dataSources (and not only the one on metadata
version in RAM)
Fixes#9827
Also uncovered a conflict with `@objectType('Relation')` and
`@objectType('relation)`
I don't want to address it in this PR so I will create a followup issue
when we close this but I think there's a confusion between
Relation/RelationMetadata, it's unclear what is what
---------
Co-authored-by: Antoine Moreaux <moreaux.antoine@gmail.com>
- Sync metadata to create workflow entities, since those are not behind
a flag anymore
- Seed workflow views
- Seed workspace favorite for workflow
- Put all steps in upgrade command
In this PR:
- remove old versions upgrade commands
- add a 0.40 upgrade command to loop over all INACTIVE workspaces and
either: update to SUSPENDED (if workspaceSchema exists), update them to
SUSPENDED + deletedAt (if workspaceSchema does not exist anymore)
Note: why updating the deleted one to SUSPENDED? Because I plan to
remove INACTIVE case in the enum in 0.41
Tests made on production like database:
- dry-mode
- singleWorkspaceId
- 3 cases : suspended, deleted+suspended, deleted+suspended+delete all
data
As a follow-up of https://github.com/twentyhq/twenty/pull/9304, we are
here creating a migration to run at the next release, aiming at adding
the new aggregate operation options (CountEmpty, CountNotEmpty, ...,
PercentEmpty, PercentNotEmpty) to the enums on View and ViewField's
aggregateOperations fields.
---------
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
- Added all usable composite field types on pet custom object
- Fixed missing createdBy on people and company seeds
- DEFAULT_SUBDOMAIN is now used for login (could be improved for multi
workspace)
- Refactored ObjectMetadataStandardIdToIdMap to disambiguate from
ObjectMetadataMap
- Refactored seedCustomObjects
Fixes https://github.com/twentyhq/twenty/issues/8810
Fixes https://github.com/twentyhq/twenty/issues/5268
Fixes https://github.com/twentyhq/twenty/issues/8971
- Fixing Task/Note creation not sending position during creation
- Adding a command to backfill position being null, using existing
backfill command.
- Removed unused backfill job.
- Updated workspace entities to set position non-nullable and set a
default value to make it non-required on the API
- Updated position factory to set a default position for all objects
having a POSITION field instead of only company/people
- Moved the try/catch in each resolver factory calling
GraphqlQueryRunnerException handler, makes more sense to call it in the
actual graphql-query-runner and removing some duplicate codes
- Adding validations for input in QueryRunnerArgs factories
- Allow sync-metadata to override and sync defaultValues for certain
field types (that can't be updated by users)
- Removing health-check from sync-metadata command during force mode to
improve performances
