From 61a9c4dded77eb4f900089bf6a74e7367848c513 Mon Sep 17 00:00:00 2001 From: Abdul Fatir Date: Thu, 27 Jun 2024 19:26:07 +0200 Subject: [PATCH] Update README.md with dataset and evaluation details (#136) *Description of changes:* This PR updates README.md with dataset and evaluation details By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. --- README.md | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 2c851c8..4c5e3c2 100644 --- a/README.md +++ b/README.md @@ -11,12 +11,13 @@ ## ๐Ÿš€ News +- **27 June 2024**: ๐Ÿš€ [Released datasets](https://huggingface.co/datasets/autogluon/chronos_datasets) used in the paper and an [evaluation script](./scripts/README.md#evaluating-chronos-models) to compute the WQL and MASE scores reported in the paper. - **17 May 2024**: ๐Ÿ› Fixed an off-by-one error in bin indices in the `output_transform`. This simple fix significantly improves the overall performance of Chronos. We will update the results in the next revision on ArXiv. - **10 May 2024**: ๐Ÿš€ We added the code for pretraining and fine-tuning Chronos models. You can find it in [this folder](./scripts/training). We also added [a script](./scripts/kernel-synth.py) for generating synthetic time series data from Gaussian processes (KernelSynth; see Section 4.2 in the paper for details). Check out the [usage examples](./scripts/). - **19 Apr 2024**: ๐Ÿš€ Chronos is now supported on [AutoGluon-TimeSeries](https://auto.gluon.ai/stable/tutorials/timeseries/index.html), the powerful AutoML package for time series forecasting which enables model ensembles, cloud deployments, and much more. Get started with the [tutorial](https://auto.gluon.ai/stable/tutorials/timeseries/forecasting-chronos.html). - **08 Apr 2024**: ๐Ÿงช Experimental [MLX inference support](https://github.com/amazon-science/chronos-forecasting/tree/mlx) added. If you have an Apple Silicon Mac, you can now obtain significantly faster forecasts from Chronos compared to CPU inference. This provides an alternative way to exploit the GPU on your Apple Silicon Macs together with the "mps" support in PyTorch. -- **25 Mar 2024**: [v1.1.0 released](https://github.com/amazon-science/chronos-forecasting/releases/tag/v1.1.0) with inference optimizations and `pipeline.embed` to extract encoder embeddings from Chronos. -- **13 Mar 2024**: Chronos [paper](https://arxiv.org/abs/2403.07815) and inference code released. +- **25 Mar 2024**: ๐Ÿš€ [v1.1.0 released](https://github.com/amazon-science/chronos-forecasting/releases/tag/v1.1.0) with inference optimizations and `pipeline.embed` to extract encoder embeddings from Chronos. +- **13 Mar 2024**: ๐Ÿš€ Chronos [paper](https://arxiv.org/abs/2403.07815) and inference code released. ## โœจ Introduction @@ -144,9 +145,13 @@ context = torch.tensor(df["#Passengers"]) embeddings, tokenizer_state = pipeline.embed(context) ``` -### Pretraining and fine-tuning +### Pretraining, fine-tuning and evaluation -Scripts for pretraining and fine-tuning Chronos models can be found in [this folder](./scripts/). +Scripts for pretraining, fine-tuning and evaluating Chronos models can be found in [this folder](./scripts/). + +## :floppy_disk: Datasets + +Datasets used in the Chronos paper for pretraining and evaluation (both in-domain and zero-shot) are available through the HuggingFace repos: [`autogluon/chronos_datasets`](https://huggingface.co/datasets/autogluon/chronos_datasets) and [`autogluon/chronos_datasets_extra`](https://huggingface.co/datasets/autogluon/chronos_datasets_extra). Check out these repos for instructions on how to download and use the datasets. ## ๐Ÿ”ฅ Coverage