Commit graph

24 commits

Author SHA1 Message Date
martmull
16e3e38b79
Improve getting started doc (#19138)
- improves
`packages/twenty-docs/developers/extend/apps/getting-started.mdx`

---------

Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-04-01 20:39:44 +00:00
Charles Bochet
191a277ddf
fix: invalidate rolesPermissions cache + add Docker Hub auth to CI (#19044)
## Summary

### Cache invalidation fix
- After migrating object/field permissions to syncable entities (#18609,
#18751, #18567), changes to `flatObjectPermissionMaps`,
`flatFieldPermissionMaps`, or `flatPermissionFlagMaps` no longer
triggered `rolesPermissions` cache invalidation
- This caused stale permission data to be served, leading to flaky
`permissions-on-relations` integration tests and potentially incorrect
permission enforcement in production after object permission upserts
- Adds the three permission-related flat map keys to the condition that
triggers `rolesPermissions` cache recomputation in
`WorkspaceMigrationRunnerService.getLegacyCacheInvalidationPromises`
- Clears memoizer after recomputation to prevent concurrent
`getOrRecompute` calls from caching stale data

### Docker Hub rate limit fix
- CI service containers (postgres, redis, clickhouse) and `docker
run`/`docker build` steps were pulling from Docker Hub
**unauthenticated**, hitting the 100-pull-per-6-hour rate limit on
shared GitHub-hosted runner IPs
- Adds `credentials` blocks to all service container definitions and
`docker/login-action` steps before `docker run`/`docker compose`
commands
- Uses `vars.DOCKERHUB_USERNAME` + `secrets.DOCKERHUB_PASSWORD`
(matching the existing twenty-infra convention)
- Affected workflows: ci-server, ci-merge-queue, ci-breaking-changes,
ci-zapier, ci-sdk, ci-create-app-e2e, ci-website,
ci-test-docker-compose, preview-env-keepalive, spawn-twenty-docker-image
action
2026-03-27 17:32:53 +01:00
Félix Malfait
dd84ab25df
chore: optimize app-dev Docker image and add CI test (#18856)
## Summary

- **Reduce app-dev image size** by stripping ~60MB of build artifacts
not needed at runtime from the server build stage: `.js.map` source maps
(29MB), `.d.ts` type declarations (9MB), compiled test files (14MB), and
unused package source directories (~9MB).
- **Add CI smoke test** for the `twenty-app-dev` all-in-one Docker
image, running in parallel with the existing docker-compose test. Builds
the image, starts the container, and verifies `/healthz` returns 200.

## Test plan

- [x] Built image locally and verified server, worker, Postgres, and
Redis all start correctly
- [x] Verified `/healthz` returns 200 and frontend serves at `/`
- [ ] CI `test-compose` job passes (existing test, renamed from `test`)
- [ ] CI `test-app-dev` job passes (new parallel job)

Made with [Cursor](https://cursor.com)
2026-03-24 08:44:30 +01:00
Charles Bochet
ef499b6d47
Re-enable disabled lint rules and right-size CI runners (#18461)
## Summary

- Re-enable one lint rule that was temporarily disabled during the
ESLint-to-Oxlint migration:
- **`twenty/sort-css-properties-alphabetically`** in twenty-front — 578
violations auto-fixed across 390 files
- Document why **`typescript/consistent-type-imports`** cannot be
auto-fixed in twenty-server: NestJS relies on `emitDecoratorMetadata`
for DI, so converting constructor parameter imports to `import type`
erases them at compile time and breaks dependency injection at runtime
- Right-size CI runners, reducing 8-core usage from 18 jobs to 3:

| Change | Jobs | Rationale |
|--------|------|-----------|
| **Keep 8-core** | `ci-merge-queue/e2e-test`,
`ci-front/front-sb-build`, `ci-front/front-build` | Heavy builds needing
max CPU + memory (10GB NODE_OPTIONS, full Storybook webpack bundling) |
| **8-core → 4-core** | `ci-server` (build, lint-typecheck, validation,
test, integration-test), `ci-front/front-sb-test`,
`ci-zapier/server-setup`, `ci-sdk/sdk-e2e-test` | Already sharded into
10-12 parallel instances, I/O-bound (DB/Redis), or moderate single
builds |
| **8-core → 2-core** | `ci-emails/emails-test` | Trivially lightweight
(build + curl health check) |
| **Removed** | `ci-front/front-chromatic-deployment` | Dead code —
permanently disabled with `if: false` |

- Fix merge queue CI issues:
- **Concurrency**: Use `merge_group.base_ref` instead of unique merge
group ref so new queue entries cancel previous runs
- **Required status checks**: Add `merge_group` trigger to all 6
required CI workflows (front, server, shared, website, docker-compose,
sdk) with `changed-files-check` auto-skipped for merge_group events —
status check jobs auto-pass without re-running full CI
- **Build caching**: Add Nx build cache restore/save to E2E test job
with fallback to `main` branch cache for faster frontend and server
builds

## Test plan

- [ ] CI passes on this PR (verifies lint rule auto-fix works)
- [ ] Verify 4-core runner jobs complete within their 30-minute timeouts
- [ ] Verify merge queue status checks auto-pass (ci-front-status-check,
ci-server-status-check, etc.)
- [ ] Verify merge queue E2E concurrency cancels previous runs when a
new PR enters the queue
2026-03-06 13:33:02 +00:00
Charles Bochet
d37ed7e07c
Optimize merge queue to only run E2E and integrate prettier into lint (#18459)
## Summary

- **Merge queue optimization**: Created a dedicated
`ci-merge-queue.yaml` workflow that only runs Playwright E2E tests on
`ubuntu-latest-8-cores`. Removed `merge_group` trigger from all 7
existing CI workflows (front, server, shared, website, sdk, zapier,
docker-compose). The merge queue goes from ~30+ parallel jobs to a
single focused E2E job.
- **Label-based merge queue simulation**: Added `run-merge-queue` label
support so developers can trigger the exact merge queue E2E pipeline on
any open PR before it enters the queue.
- **Prettier in lint**: Chained `prettier --check` into `lint` and
`prettier --write` into `lint --configuration=fix` across `nx.json`
defaults, `twenty-front`, and `twenty-server`. Prettier formatting
errors are now caught by `lint` and fixed by `lint:fix` /
`lint:diff-with-main --configuration=fix`.

## After merge (manual repo settings)

Update GitHub branch protection required status checks:
1. Remove old per-workflow merge queue checks (`ci-front-status-check`,
`ci-e2e-status-check`, `ci-server-status-check`, etc.)
2. Add `ci-merge-queue-status-check` as the required check for the merge
queue
2026-03-06 13:20:57 +01:00
Charles Bochet
d48c58640c
Migrate CI runners from Depot back to GitHub-hosted runners (#18347)
## Summary
- Replaces all `depot-ubuntu-24.04` runners with `ubuntu-latest`
- Replaces all `depot-ubuntu-24.04-8` runners with
`ubuntu-latest-8-cores`
- Updates storybook build cache keys in ci-front.yaml to reflect the
runner name change

Reverts the temporary Depot migration introduced in #18163 / #18179
across all 23 workflow files.
2026-03-03 14:14:27 +01:00
Charles Bochet
129d1ede86
Change runners temp (#18163)
Temporarily moving all ubuntu-latest 1 core to depot except ci-website
2026-02-23 10:53:31 +01:00
Marie
ceba0972cd
Remove tests run on push on main (#16971)
Since tests are now run in the pre-merge queue with the latest main
version, they need not to be run again when merged into main, it would
be the exact same thing
2026-01-06 17:53:08 +00:00
Félix Malfait
e6491d6a80
feat(i18n): fix translation QA issues and add automation (#16756)
## Summary

This PR fixes translation QA issues and adds automation to prevent
future issues.

### Translation Fixes
- Fixed **escaped Unicode sequences** in translations (e.g.,
`\u62db\u5f85` → `招待`)
- Removed **corrupted control characters** from .po files (null bytes,
invalid characters)
- Fixed **missing/incorrect placeholders** in various languages
- Deleted **35 problematic translations** via Crowdin API that had
variable mismatches

### New Scripts (in `packages/twenty-utils/`)
- `fix-crowdin-translations.ts` - Auto-fixes encoding issues and syncs
to Crowdin
- `fix-qa-issues.ts` - Fixes specific QA issues via Crowdin API
- `translation-qa-report.ts` - Generates weekly QA report from Crowdin
API

### New Workflow
- `i18n-qa-report.yaml` - Weekly workflow that creates a PR with
translation QA issues for review

### Other Changes
- Moved GitHub Actions from `.github/workflows/actions/` to
`.github/actions/`
- Fixed `date-utils.ts` to avoid nested `t` macros in plural expressions
(root cause of confusing placeholders)

### QA Status After Fixes
| Category | Count | Status |
|----------|-------|--------|
| variables | 0  | Fixed |
| tags | 1 | Minor |
| empty | 0  | Fixed |
| spaces | 127 | Low priority |
| numbers | 246 | Locale-specific |
| special_symbols | 268 | Locale-specific |
2025-12-22 17:30:46 +01:00
Félix Malfait
fc6b136c2f
fix: resolve GitHub Actions security vulnerabilities (#16174)
## 🔒 Security Fixes

This PR addresses security vulnerabilities identified by GitHub CodeQL
security scanning.

### Changes

#### 1. Fix Shell Command Injection (High Severity)
**File:** `.github/workflows/docs-i18n-pull.yaml`

**Issue:** Direct interpolation of `${{ github.head_ref }}` in shell
command was susceptible to command injection attacks.

**Fix:** Assign GitHub context variable to environment variable first:
```yaml
run: |
  git push origin "HEAD:$HEAD_REF"
env:
  HEAD_REF: ${{ github.head_ref }}
```

This prevents malicious input from being executed as shell commands.

#### 2. Add Missing Workflow Permissions (Medium Severity)
**File:** `.github/workflows/ci-test-docker-compose.yaml`

**Issue:** Workflow did not explicitly define GITHUB_TOKEN permissions,
running with overly broad defaults.

**Fix:** Added explicit minimal permissions:
```yaml
permissions:
  contents: read
```

This applies to all 3 jobs in the workflow:
- `changed-files-check`
- `test`
- `ci-test-docker-compose-status-check`

### Security Impact

-  Prevents potential shell injection attacks via pull request branch
names
-  Follows principle of least privilege for GitHub Actions tokens
-  Aligns with GitHub Actions security best practices
-  Resolves all CodeQL security alerts for these workflows

### References

- [GitHub Actions: Security hardening for GitHub
Actions](https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions)
- [GitHub Actions: Permissions for the
GITHUB_TOKEN](https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token)
- Related attacks: 2025 Nx supply chain attack, 2024 ultralytics/actions
attack
2025-11-28 13:15:33 +01:00
Félix Malfait
5375a478db
Fix: Make CI .env manipulation robust against missing trailing newlines (#15189)
## Problem

CI workflow started timing out on October 14, 2025 after commit
`d750df7fff` removed the trailing newline from `.env.example`.

## Root Cause

When `.env.example` lacks a trailing newline:
```bash
# Last line without newline
# CLICKHOUSE_URL=...twenty
```

And CI runs:
```bash
echo "NODE_PORT=3002" >> .env
```

Result:
```bash
# CLICKHOUSE_URL=...twentyNODE_PORT=3002  ← Commented out!
```

Server starts on default port 3000 instead of 3002, health check fails.

## Fix

1. **Restore trailing newline** to `.env.example`
2. **Make all CI `.env` operations robust** by adding `echo "" >> .env`
before appending
3. **Simplified `set_env_var`** function to always add newline first

Now works regardless of whether template files have trailing newlines.

## Files Changed

- 6 CI workflow files
- 1 .env.example file
2025-10-18 13:46:56 +02:00
Félix Malfait
d76abefdee
Fix CI concurrency: prevent test cancellation on main branch (#15188)
## Problem
The concurrency rules in CI workflows were cancelling in-progress test
runs even on the main branch. This caused inconsistent check counts when
multiple commits were pushed in quick succession.

## Solution
Updated `cancel-in-progress` in all CI workflows to be conditional:
- **On main branch**: Tests run to completion (no cancellation)
- **On feature branches**: Tests are cancelled when new commits are
pushed (saves CI resources)

## Changes
Modified 11 workflow files to use:
```yaml
cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}
```

This ensures every commit to main gets fully tested while maintaining
efficiency on feature branches.
2025-10-18 10:50:54 +02:00
Paul Rastoin
f2691e53a0
[CI]: Increase status check timeout (#11896)
# Introduction
Some jobs start to fail because they exceed the timeout
Such as this job
https://github.com/twentyhq/twenty/actions/runs/14862320942/job/41730200276?pr=11881
2025-05-06 17:14:58 +02:00
Charles Bochet
c6e5238d71
Fix Docker compose CI to stop building postgres-spilo image (#10595)
As a follow up of #10594
2025-02-28 20:55:34 +01:00
John Sessford
25cb909e17
Improve Docker-Compose Install Experience (#9781)
This PR updates the docker-compose installation documentation and env
sample to improve the setup experience.

- Updates the URLs for raw files to reference main branch, which is
likely where new users will be pulling from initially. This seems to be
the most straightforward option; assume that advanced users who want to
retrieve it from a particular tag will know to change the URL for their
scenario.
- Fixes an improperly stated curl command.
- Adds a note that the PGPASSWORD_SUPERUSER should be URL-safe. This is
required since the value is later concat into a PG_DATABASE_URL as a
URL, and expected to be in proper URL format. Touches on #8597.

---------

Co-authored-by: Félix Malfait <felix@twenty.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
2025-01-24 16:13:28 +01:00
Paul Rastoin
560f715c37
[CI][NITPICK]: Rename prerequisites jobs to changed-files-check (#9670)
Related to https://github.com/twentyhq/twenty/pull/9643

Renaming `prerequisites` jobs to a more accurate `changed-files-check`
2025-01-16 14:09:59 +01:00
Paul Rastoin
f8ddc02b8e
[CI] Refactor changed files integration (#9643)
# BEFORE
[run](https://github.com/twentyhq/twenty/actions/runs/12806801953)

![image](https://github.com/user-attachments/assets/f0a8ffe3-3fc0-42ca-b2ee-8a980606b5dd)
# AFTER
[run](https://github.com/twentyhq/twenty/actions/runs/12807034402)

![image](https://github.com/user-attachments/assets/5117c680-6804-416b-a8c8-bf00614ca453)

## Motivations:
- less workflow to whitelist as blocking for PRs
- less if condition per step

cons:
- quite verbose
- need to manually sync the `ci-NAME-status-check` needs list to any
other existing and should be dep jobs

## Version migration
Migrated to the latest `changed-files@45` version, getting rid of the
`set-output` usage warnings

## Tests runs:

With mutation:
- [Success
flow](https://github.com/twentyhq/twenty/actions/runs/12791958651/job/35661546343)
- [server-setup failure
flow](https://github.com/twentyhq/twenty/actions/runs/12792225779)
- [Other job failure
flow](https://github.com/twentyhq/twenty/actions/runs/12792313463), one
of the `inner` job failed
- [Manual cancel
flow](https://github.com/twentyhq/twenty/actions/runs/12792313463)
`ci-server-status-check` also has the `cancelled` status
- [Matrix
failure](https://github.com/twentyhq/twenty/actions/runs/12806883553)

Without mutation:
- [Nothing to do
flow](https://github.com/twentyhq/twenty/actions/runs/12792098384),
skipped `inner` job but `ci-server-status-check` still succeeded

## Notes
### Linter
We should setup a `yml` prettier and linter for the `.github/worfklows`
folder
### Centralized `ci-NAME-status-check` logic
Unfortunately I couldn't achieve to either make a `composite` action or
a `reusable-workflow`, as I could not access the correct layer to run
the `always` but also acessing the `needs` context
2025-01-16 13:37:28 +01:00
Félix Malfait
736635a94b
Begin moving to postgres spilo + adding pgvector (#8309)
We will remove the `twenty-postgres` image that was used for local
development and only use `twenty-postgres-pilo` (which we use in prod),
bringing the development environment closer to prod and avoiding having
to maintain 2 images.


Instead of provisioning the super user after the db initialization, we
directly rely on the superuser provided by Spilo for simplicity. We also
introduce a change that tries to create the right database (`default` or
`test`) based on the context.
  

How to test:
```
docker build -t twentycrm/twenty-postgres-spilo:latest -f ./packages/twenty-docker/twenty-postgres-spilo/Dockerfile .
docker images --no-trunc | grep twenty-postgres-spilo
postgres-on-docker:
	docker run \
	--name twenty_pg \
	-e PGUSER_SUPERUSER=twenty \
	-e PGPASSWORD_SUPERUSER=twenty \
	-e ALLOW_NOSSL=true \
	-v twenty_db_data:/home/postgres/pgdata \
	-p 5432:5432 \
	REPLACE_WITH_IMAGE_ID
```
2024-11-15 09:38:30 +01:00
Félix Malfait
a7a7d62502
Fix CI timeout (#8408) 2024-11-08 13:53:38 +01:00
Félix Malfait
d8831512d4
Add CI timeouts and enable playwright (#8405) 2024-11-08 12:24:22 +01:00
ZiaCodes
57d9b8e8b4
feat: generate secret function and replaced few instances (#7810)
This PR fixes #4588

---------

Co-authored-by: Félix Malfait <felix@twenty.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
2024-10-30 12:07:11 +01:00
shubham yadav
b914182b78
Update workflows to optimize CI processes (#7828)
This Pull Request addresses the need to optimize our Continuous
Integration (CI) workflows for Playwright tests and release processes.
The changes implemented aim to reduce unnecessary resource usage by
conditionally executing jobs based on relevant file changes and
Implement https://github.com/tj-actions/changed-files step

## Changes logs
- Updated `ci-test-docker-compose.yaml , ci-chrome-extension.yaml ` to
check for changed files before running tests.
- Updated `ci-front.yaml , ci-utils.yaml , ci-website.yaml ,
ci-server.yaml` to check for changed files before running tests.
- Enhanced `playwright.yml` to skip unnecessary tests based on file
changes.
2024-10-21 12:02:14 +02:00
Quentin G
bf60227d67
feat: add SENTRY_RELEASE env (#4912)
Add support for a new SENTRY_RELEASE and SENTRY_ENVIRONMENT env.
It is optional and allows to init sentry with a Release version and an
env (used internally at Twenty).
Docker image have been updated do intergrate the new env as an Argument
2024-04-11 16:53:15 +02:00
Quentin G
ccd02fe58c
feat: init docker compose test (#4730)
Job testing if docker compose is working. It triggers on pull_requests.
2024-04-03 18:08:21 +02:00