angular/aio/scripts/generate-events

Generating data for angular.io/events

This document and the contents of this directory contain information and source code related to generating data for the angular.io Events page.

Directory contents

The following list gives a brief description of the contents of this directory and their purpose. For more details see the following sections.

• apps-script-extension/: The source code for the Apps Script extension that needs to be added to the Google Sheet spreadsheet.
• index.mjs: The script for retrieving the data from the Firebase database and generating the events.json file for angular.io.

Background

The "Events" page on angular.io has two sections: One for upcoming events and one for past events.

Originally, the events were hard-coded into the page's HTML, which meant that the page had to be updated twice for each event (once to add it to the list of upcoming events and once more to move it to the list of past events).

Later, the setup was changed so that the events were loaded as JSON and passed to an EventsComponent, which was able to categorize them as "upcoming" or "past" based on the date. This reduced the maintenance overhead by only requiring one update per event (just to add it to the events.json file that was part of the angular.io source code).

However, since the DevRel team had to maintain a separate list of events outside angular.io (in a more suitable format for their needs), that setup still required unnecessary work and resulted in having to manually duplicate the data in two places. Additionally, due to the extra overhead of updating the events list on angular.io (creating a pull request, getting it approved, merged and finally deployed), the events page was often out of date.

This document describes the latest, revised process for generating the events data with the aim of:

• Minimizing the manual overhead.
• Avoiding data duplication.
• Ensuring the freshness of the data on angular.io.
• Minimizing changes to the current DevRel team workflow.

The current process

This section describes the current setup and process for generating events data for angular.io.

Overview

In a nutshell, the setup can be summarized as follows:

1. The DevRel team keeps information about events in a Google Sheets spreadsheet (in the appropriate format).
2. An Apps Script extension on the spreadsheet periodically saves the relevant information (such as event names and dates) in a Firebase Realtime Database.
3. There is a script which can query the database and generate events.json based on the latest data.
4. A GitHub Actions periodically runs the script and creates a pull request if there are updates.

Apps Script extension overview

In a nutshell, the Apps Script extension works as follows:

1. An onEdit trigger is invoked every time the spreadsheet is edited and checks whether a team allocation sheet was edited. If so, it adds the name of the sheet to a list of edited sheets.
2. A time-driven trigger is invoked periodically and checks to see if there are any sheets that have been edited since the last invocation. If so, it extracts the event data from each edited sheet and updates the Firebase database.

Useful resource: https://stackoverflow.com/questions/53207906/how-to-integrate-firebase-into-google-apps-script-without-using-deprecated-dat#answer-53211786

How to set up

1. Have a Google Sheets spreadsheet for keeping event information. The spreadsheet must follow some format requirements in order for the script to be able to extract event information. Look at the source code in apps-script-extension/ for details, but the main requirements are:

• There should be a sheet named XXXX Team Allocation for each year, where XXXX is the year (for example, 2022 Team Allocation).
• Each team allocation sheet should have the event dates on the first row (potentially after some empty cells) and the dates should be displayed in the format M/D (for example, 4/11 for April 11th).
• Each team allocation sheet should have the event names on the second row (each under the corresponding date cell). Event names can optionally be links pointing to the event's web page.

NOTE: For this project, we use our Angular Conferences Speaker Tracker spreadsheet.

2. Create an Apps Script extension for the aforementioned spreadsheet with the source code from the apps-script-extension/ directory. To do this, open the spreadsheet, click on Extensions > Apps Script, create the necessary files as seen in the source code (with the difference that the .js extension must be replaced with .gs) and copy the source code. For appsscript.json, follow the instructions here to make it appear in the in-browser editor.

3. Set up a time-driven trigger to run the updateEventsOnFirebase() function (found in persister.gs). Adjust the frequency according to your needs.

4. Have a Firebase project with Realtime Database enabled.

5. Follow the instructions here to set up security rules and make sure they are deployed to the Firebase project as needed. You can see the database security rules used for this project in database.rules.json. These rules will allow anyone to read the events from the database, but only someone with access to the Firebase project will be able to update the events in the database.

NOTE: For this project, the rules are deployed as part of the deploy_aio CI job.

6. Ensure that the account that was used to create the Google Sheets trigger on step 3 also has access to the Firebase project (otherwise, the trigger will fail to update the database when events change in the spreadsheet).

7. Wire the index.mjs script to run when necessary to generate an updated events.json file.

NOTE: For this project, there is a GitHub Action that periodically runs the script and creates a pull request (if necessary).

8. Ensure that both constants.gs and index.mjs point to the correct database URL.

How to update

Although the source code in apps-script-extension/ and the actual code used in the spreadsheet are independent, it is advised for versioning purposes to keep the two in sync. Whenever a change is needed to be made to the Apps Script extension, the change should be applied in both places.

How to debug

If the process does not work as expected, i.e. changes are made to the spreadsheet but no pull request is created to update event.json, there are three places where one can look to find out at which point the process breaks:

1. Ensure that the Apps Script extension runs successfully:

• You can check the previous executions of the updateEventsOnFirebase() function by opening the spreadsheet and clicking on Extensions > Apps Script > Executions. There, you can see when the extension ran and also click on an execution to see the output log.

  NOTE: For this project, executions can be found here.

• You can also manually run the updateEventsOnFirebase() function by opening the spreadsheet and clicking on Extensions > Apps Script > Editor > persister.gs. From there, choose the function in the dropdown at the top of the editor and click the Run button.

  NOTE: For this project, the editor can be found here.

2. Ensure that the database contains the correct data:

• Visit the Firebase console, open the targeted project and click on Realtime Database.

• From there, you can explore the data and ensure that the events node contains the expected data.

NOTE: For this project, the database can be found here.

3. If you have set up a GitHub Action, ensure that it runs successfully:

• You can check the previous runs of the GitHub action by visiting the repository page on GitHub, clicking the Actions tab and then choosing the correct workflow from the list on the left.

• From there, you can see when the action was run and also click on a run to see the completion status of each job and the output log.

• You can also manually run the action, by clicking the Run workflow button.

NOTE: For this project, you can see runs of the Update AIO events GitHub Action here.

Trade-offs/Alternatives considered

This section describes trade-offs made and alternative implementations/variations that were considered.

Trade-offs:

• The current implementation provides minimal data for each event (name, start date and optionally link to web site). Specifically, compared to the previous implementation, data that was dropped includes: end dates, workshop dates (when applicable), mandatory link to web site and optional tooltips (to be shown on hover). This decision was made in order to be able to keep the existing workflow/data layout of the DevRel spreadsheet and may be revisited in the future.

Alternatives considered:

• We considered using Cloud Firestore, which is Firebase's newer database offering, but communicating with it via REST seemed more involved than with the Realtime Database.

• We considered updating the database on each edit, since this would avoid the need for a time-driven trigger. This proved to be problematic for the following reasons:

  • It is very inefficient to do without debouncing (as there could be multiple consecutive updates in a short time).
  • Debouncing doesn't seem to be possible in Apps Script extensions without using installable triggers. See below on why installable triggers proved problematic as well.

• We considered reducing the setup requirements by programmatically adding an installable trigger to update the database shortly after an edit. This proved problematic, because it would require all users with write access to the spreadsheet to also:

  • Grant authorization for the firebase.database, script.external_request and script.scriptapp OAuth scopes to the extension.
  • Have access to the Firebase project.

  We decided it would be more "ergonomic" to have someone (for example, the DevRel lead) with access to the Firebase project manually create a time-driven installable trigger. This entails an extra step, but is a one-time action, so the overhead is minimal.