*Issue #, if available:*
*Description of changes:* Adds support for custom callbacks after each
batch is processed during prediction. This allows for keeping track of
the time limit in AutoGluon.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:*
*Description of changes:* This PR improves test coverage by adding unit
tests for `df_utils`. Previously these methods were only being tested as
part of Chronos-2 integration tests.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:*
*Description of changes:*
- Rename `predict_batches_jointly` to `cross_learning`
- Add deprecation warning
- Add cross_learning to predict_df docstring
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:* Fixes#403
*Description of changes:*
- Update the `future_df` validation logic to only check that
`prediction_length` values are provided for each item.
- Update unit tests for DF-based methods in `test_chronos2.py`
- Ignore fine-tuned checkpoint folders with `.gitignore`
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:*
*Description of changes:* This PR adds a `validate_inputs ` argument to
`predict_df` (defaults to `True`), which allows the user to disable
dataframe validation when they know that their dataframe is in the right
format. This reduces runtime by removing the input validation component,
e.g., when calling this method from
[AutoGluon](https://github.com/autogluon/autogluon/pull/5427), and also
handles series with shorter than 3 timesteps.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:*
*Description of changes:* Adds support for LoRA fine-tuning.
- [x] Move peft/pandas dependency to an extra
- [x] Add tests for LoRA
- [x] Update notebook with LoRA info
- [x] Enable automatic recognition and loading of LoRA adapters
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:* #354
*Description of changes:* This PR adds `Chronos2Pipeline.embed` to
enable users to extract embeddings from the last encoder layer in an
easy way. The API and behavior is similar to what Chronos and
Chronos-Bolt provides.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:*
*Description of changes:* This PR adds `predict_df` to the base pipeline
which enables pandas support for the univariate Chronos and Chronos-Bolt
models.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:*
*Description of changes:*
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
This pull request introduces configurable attention backends to the
Chronos-2 model, allowing users to select between eager, SDPA, and
FlashAttention-2 implementations.
---------
Co-authored-by: Oleksandr Shchur <oleks.shchur@gmail.com>
Co-authored-by: Abdul Fatir <Abdulfatirs@gmail.com>
*Issue #, if available:*
*Description of changes:* This PR adds the Chronos-2 model.
* Chronos-2 modeling and pipeline code, including tests.
* Updated `pyproject.toml`. Merge `training` and `evaluation` extras
into a single `dev` extra. This stuff is only relevant for the Chronos
models.
* Added `predict_fev` to `BaseChronosPipeline`.
* Changes to `InstanceNorm` for Chronos-Bolt to make it general and
compatible with Chronos-2.
* Minor renaming and polishing in the inference code for Chronos and
Chronos-Bolt.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
---------
Co-authored-by: Oleksandr Shchur <oleks.shchur@gmail.com>
*Issue #, if available:* On Linux, the final call to `.to` creates
trouble when input tensors are integer. For example:
```
>>> a = torch.tensor([1])
>>> b = torch.stack([torch.full((1,), torch.nan), a])
>>> b
tensor([[nan],
[1.]])
>>> b.to(a)
tensor([[-9223372036854775808],
[ 1]])
```
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:* N/A
*Description of changes:* This PR ensures that predictions are returned
in FP32 and on the CPU device. This choice is now better because we have
two types of models which have different types of forecasts (samples vs.
quantiles). Furthermore, `int64` input_type (our README example is one
such case) ran into issues with `predict_quantiles` before. This choice
also fixes that.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
---------
Co-authored-by: Abdul Fatir Ansari <ansarnd@amazon.de>
*Issue #, if available:* N/A
*Description of changes:* This PR adds support for Chronos-Bolt models.
TODOs:
- [x] Update evaluation script
- [x] Fix and add tests for Bolt
- [x] Update docstrings
- [x] Update README example and mention Chronos-Bolt
- [x] Update results bar plot in README
- [x] Add versions for libraries in `pyproject.toml`
- [x] Check that the training and eval scripts work
- [x] Change `autogluon` -> `amazon` in model names
Post Merge:
- [ ] Update Citation style in README, both Github and HuggingFace repos
- [ ] Remove note about AutoGluon
- [ ] Update READMEs of original Chronos models to refer to Chronos-Bolt
NOTE: To be merged after Chronos-Bolt models are available under the
`amazon` namespace on HF.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
---------
Co-authored-by: Abdul Fatir Ansari <ansarnd@amazon.de>
Co-authored-by: Caner Turkmen <turkmen.ac@gmail.com>
Co-authored-by: Lorenzo Stella <stellalo@amazon.com>
*Issue #, if available:* Fixes#193
*Description of changes:* Passing in contexts in lower precision than
float32 may result in a drop of accuracy. This change ensures that the
tokenizer (which does scaling and quantization) operates on a float32
batch.
Tested across GPU/CPU and different context dtypes with
```python
from itertools import product
import pandas as pd
import torch
from chronos import ChronosPipeline
import matplotlib.pyplot as plt # requires: pip install matplotlib
import numpy as np
df = pd.read_csv("https://raw.githubusercontent.com/AileenNielsen/TimeSeriesAnalysisWithPython/master/data/AirPassengers.csv")
for context_dtype, context_device, model_dtype, model_device in product(
[torch.bfloat16, torch.float16, torch.float32],
["cpu"], # only cpu input supported at the moment
[torch.bfloat16, torch.float16, torch.float32],
["cpu", "cuda"],
):
pipeline = ChronosPipeline.from_pretrained(
"amazon/chronos-t5-tiny",
device_map=model_device,
torch_dtype=model_dtype,
)
forecast = pipeline.predict(
context=torch.tensor(df["#Passengers"]).to(dtype=context_dtype, device=context_device),
prediction_length=65,
num_samples=20,
limit_prediction_length=False,
)
assert forecast.dtype == context_dtype, f"{forecast.dtype=} but {context_dtype=}"
assert str(forecast.device) == context_device, f"{forecast.device=} but {context_device=}"
forecast_index = range(len(df), len(df) + 65)
low, median, high = np.quantile(forecast[0].to(device="cpu", dtype=torch.float32).numpy(), [0.1, 0.5, 0.9], axis=0)
plt.figure(figsize=(8, 4))
plt.plot(df["#Passengers"], color="royalblue", label="historical data")
plt.plot(forecast_index, median, color="tomato", label="median forecast")
plt.fill_between(forecast_index, low, high, color="tomato", alpha=0.3, label="80% prediction interval")
plt.legend()
plt.grid()
plt.show()
```
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
Fixes https://github.com/amazon-science/chronos-forecasting/issues/181.
Chronos' tokenizer has a vocabulary size of `n_tokens`. Among these,
there are `n_special_tokens` reserved for EOS, PAD, etc. and `n_tokens -
n_special_tokens` allocated to numerical values. However, the provided
`MeanScaleUniformBins` tokenizer creates` n_tokens - n_special_tokens +
1` different buckets, resulting in a total of `n_tokens + 1` possible
tokens. This causes training and inference errors when one of the data
points gets allocated to the largest bucket, as the model requires 0 <=
token_id < n_tokens.
This PR modifies the `MeanScaleUniformBins` tokenizer, so that it
creates one less bucket for numerical values.
---
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
---------
Co-authored-by: Lorenzo Stella <lorenzostella@gmail.com>
*Description of changes:* This splits `input_transform` into
`context_input_transform` and `label_input_transform`. Previously,
`input_transform` was being used for both context and label during
training which would lead to incorrect results where `prediction_length`
> `context_length`.
TODO:
- [x] Update docstrings
- [x] Test the training script
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
---------
Co-authored-by: Abdul Fatir Ansari <ansarnd@amazon.com>
*Description of changes:*
The bin indexes were shifted by one between input transform and output
transform. Subtracting 1 to the sampled tokens in output transform lead
to the correct reconstruction of the signal.
Add a test to ensure the consistency of the Chronos Tokenizer.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
Co-authored-by: Lorenzo Stella <stellalo@amazon.com> and Abdul Fatir
Ansari <ansarnd@amazon.com>
*Description of changes:* Fix some type checking issues, add mypy to
github workflow, apply black
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Description of changes:* This PR adds `pipeline.embed` which extracts
encoder embeddings from the model. These embeddings may be useful for
some downstream tasks such as classification, so this is useful to have.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
---------
Co-authored-by: Abdul Fatir Ansari <ansarnd@amazon.de>
