*Issue #, if available:*
*Description of changes:*
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:*
*Description of changes:*
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:*
*Description of changes:* This PR adds a `validate_inputs ` argument to
`predict_df` (defaults to `True`), which allows the user to disable
dataframe validation when they know that their dataframe is in the right
format. This reduces runtime by removing the input validation component,
e.g., when calling this method from
[AutoGluon](https://github.com/autogluon/autogluon/pull/5427), and also
handles series with shorter than 3 timesteps.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:*
*Description of changes:* Adds support for LoRA fine-tuning.
- [x] Move peft/pandas dependency to an extra
- [x] Add tests for LoRA
- [x] Update notebook with LoRA info
- [x] Enable automatic recognition and loading of LoRA adapters
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:* Addresses #391
*Description of changes:*
- Speed up `convert_df_input_to_list_of_dicts_input` and
`validate_df_inputs` via a few tricks:
- Replace `df.iloc[start_idx:end_idx][col]` with
`df[col].iloc[start_idx:end_idx]` to avoid copying data on each slice
- Vectorize computation of future timestamps using numpy
- Work with `dict[str, np.ndarray]` instead of `pd.DataFrame` when
working with covariates to avoid repeated `.to_numpy()` calls.
**Before**
```
Benchmarking 20000 series, 200 steps, 0 covariates...
Average runtime: 27.33s
Benchmarking 20000 series, 200 steps, 5 covariates...
Average runtime: 44.69s
```
**After**
```
Benchmarking 20000 series, 200 steps, 0 covariates...
Average runtime: 4.60s
Benchmarking 20000 series, 200 steps, 5 covariates...
Average runtime: 8.92s
```
<details>
```python
import time
import numpy as np
import pandas as pd
from chronos.df_utils import convert_df_input_to_list_of_dicts_input
def benchmark_convert_df_input(
num_items: int, num_steps: int, num_covariates: int = 0, num_trials: int = 10, freq: str = "D"
) -> None:
"""
Benchmark convert_df_input_to_list_of_dicts_input function.
Args:
num_items: Number of time series
num_steps: Number of observations per series
num_covariates: Number of covariates to include
num_trials: Number of benchmark trials
freq: Frequency string for timestamps
"""
prediction_length = 24
# Generate context DataFrame
item_ids = np.repeat(np.arange(num_items), num_steps)
timestamps = np.tile(pd.date_range("2020-01-01", periods=num_steps, freq=freq), num_items)
df_data = {"item_id": item_ids, "timestamp": timestamps, "target": np.random.randn(num_items * num_steps)}
df_data.update({f"cov_{i}": np.random.randn(num_items * num_steps) for i in range(num_covariates)})
df = pd.DataFrame(df_data)
# Generate future_df with covariates
future_df = None
if num_covariates > 0:
future_item_ids = np.repeat(np.arange(num_items), prediction_length)
offset = pd.tseries.frequencies.to_offset(freq)
future_start = pd.Timestamp("2020-01-01") + num_steps * offset
future_timestamps = np.tile(pd.date_range(start=future_start, periods=prediction_length, freq=freq), num_items)
future_data = {"item_id": future_item_ids, "timestamp": future_timestamps}
future_data.update({f"cov_{i}": np.random.randn(num_items * prediction_length) for i in range(num_covariates)})
future_df = pd.DataFrame(future_data)
times = []
print(f"Benchmarking {num_items} series, {num_steps} steps, {num_covariates} covariates...")
for _ in range(num_trials):
start = time.perf_counter()
convert_df_input_to_list_of_dicts_input(
df=df,
future_df=future_df,
id_column="item_id",
timestamp_column="timestamp",
target_columns=["target"],
prediction_length=prediction_length,
)
end = time.perf_counter()
times.append(end - start)
print(f"Average runtime: {sum(times) / len(times):.2f}s")
if __name__ == "__main__":
# Test without covariates
benchmark_convert_df_input(20_000, 200, num_covariates=0, num_trials=1)
# Test with covariates
benchmark_convert_df_input(20_000, 200, num_covariates=5, num_trials=1)
```
</details>
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:*
*Description of changes:*
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:*
*Description of changes:*
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:*
*Description of changes:*
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:*
*Description of changes:*
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:*
*Description of changes:* Lower learning rates generally appear to be
working better. This is probably because we are doing full fine-tuning
of a model with 120M params.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:*
*Description of changes:* This PR masks rows corresponding to all
covariates in the future target. Specifically, this is to avoid the
contribution of past-only covariates in loss computation. The previous
setup was correct from the perspective of pretraining but I think this
makes more sense for fine-tuning.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:* #354
*Description of changes:* This PR adds `Chronos2Pipeline.embed` to
enable users to extract embeddings from the last encoder layer in an
easy way. The API and behavior is similar to what Chronos and
Chronos-Bolt provides.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:*
*Description of changes:* This PR adds `predict_df` to the base pipeline
which enables pandas support for the univariate Chronos and Chronos-Bolt
models.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
By default, the `transformers` library sets the `num_workers` argument
of the PyTorch DataLoader to `0`, ensuring out-of-the-box compatibility
across different platforms.
*Issue #, if available:*
*Description of changes:*
Set the DataLoader `num_workers` argument to `0` to improve
cross-platform compatibility, particularly on Windows systems where
multiprocessing requires guarded execution.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
---------
Co-authored-by: Abdul Fatir <Abdulfatirs@gmail.com>
*Issue #, if available:*
*Description of changes:*
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:*
*Description of changes:*
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
---------
Co-authored-by: Oleksandr Shchur <oleks.shchur@gmail.com>
Removed logo image from the README.
*Issue #, if available:*
*Description of changes:*
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:*
*Description of changes:*
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:*
*Description of changes:*
- Add notebook showcasing how Chronos-2 can be deployed using Amazon
SageMaker
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
Closes#345
Hi @abdulfatir
Here is the bugfix about the function
"validate_and_prepare_single_dict_task", which had 2 issue points:
1. Originally, one of this func return, the "task_n_future_covariates",
will return the ["past only" + "future known"]covariates number, by
`task_n_future_covariates = len(task_future_covariates_list)` as
`task_future_covariates_list ` is filled by for` key in
task_covariates_keys`
2. The code seems not to guarantee the last "future known" rows are
atcually what we expected, even there is a sorted option.
So, this PR fixed them by separating "past only" and "future known" covs
from the "past_covariates" input, and explicitly put the "past only"
covs rows above "future known" cov rows, supported by a temp list
"ordered_covariate_keys".
*Issue #, if available:*
*Description of changes:*
- Move the SageMaker deployment notebook to a new path to check that the
GitHub redirect feature works.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:*
*Description of changes:* The repo is classified as a Jupyter Notebook
repo. This PR attempts to fix this by counting notebooks as python files
instead of a separate thing.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:*
*Description of changes:* This PR updates `.gitattributes` to treat
notebooks as documentation rather than code when computing language
statistics. See: https://github.com/github-linguist/linguist/issues/3282
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
This pull request introduces configurable attention backends to the
Chronos-2 model, allowing users to select between eager, SDPA, and
FlashAttention-2 implementations.
---------
Co-authored-by: Oleksandr Shchur <oleks.shchur@gmail.com>
Co-authored-by: Abdul Fatir <Abdulfatirs@gmail.com>
*Issue #, if available:*
*Description of changes:*
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:*
*Description of changes:*
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:*
*Description of changes:*
Scope down GitHub token permissions for GHA security best practices.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:*
*Description of changes:*
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:*
*Description of changes:* Bump to 2.0
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:*
*Description of changes:* Update README for Chronos-2
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
---------
Co-authored-by: Oleksandr Shchur <oleks.shchur@gmail.com>
*Issue #, if available:*
*Description of changes:*
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
⚠️ Must be merged after #319 and other related PRs.
*Issue #, if available:*
*Description of changes:* Exports `chronos.__version__` and uses it
dynamically in `pyproject.toml`. Bumps version to 2.0.0.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:*
*Description of changes:* Remove `ALWAYS_DOWNLOAD` so that Chronos-2
works in a fully offline env if model files are cached.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:*
*Description of changes:* This PR adds the Chronos-2 model.
* Chronos-2 modeling and pipeline code, including tests.
* Updated `pyproject.toml`. Merge `training` and `evaluation` extras
into a single `dev` extra. This stuff is only relevant for the Chronos
models.
* Added `predict_fev` to `BaseChronosPipeline`.
* Changes to `InstanceNorm` for Chronos-Bolt to make it general and
compatible with Chronos-2.
* Minor renaming and polishing in the inference code for Chronos and
Chronos-Bolt.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
---------
Co-authored-by: Oleksandr Shchur <oleks.shchur@gmail.com>
*Issue #, if available:* Fixes#310 and closes#302
*Description of changes:* This PR fixes an issue related to the new
caching mechanism for T5 introduced in `transformers==4.54`. [Prior
versions
set](https://github.com/huggingface/transformers/blob/v4.53.3/src/transformers/models/t5/modeling_t5.py#L1328)
`encoder_config.is_encoder_decoder = False` when initializing encoder
and decoder. Following transformers, we also initialized Chronos-Bolt in
the same way. However, in v4.54 this line [has been
removed](3fd456b200/src/transformers/models/t5/modeling_t5.py (L1301))
and [new logic has been
added](3fd456b200/src/transformers/models/t5/modeling_t5.py (L494))
which relies on `is_encoder_decoder` [being
True](3fd456b200/src/transformers/models/t5/modeling_t5.py (L1007)).
This causes Chronos-Bolt to break as described in #310. This PR removes
`is_encoder_decoder = False` for both encoder and decoder which fixes
the issue. I re-ran our mini eval in the CI and got the same results for
v4.54 and v4.48 (our current lower bound).
This PR also bumps package versions.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:*
*Description of changes:*
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:*
*Description of changes:*
Just increased the max version allowed. Tests pass.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
---------
Co-authored-by: Abdul Fatir <Abdulfatirs@gmail.com>
*Issue #, if available:* See example build
https://github.com/amazon-science/chronos-forecasting/actions/runs/14302765904/job/40313421985
*Description of changes:*
- Address type-checker complaints, where possible
- Bump bugfix version of the package
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:*
*Description of changes:*
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
*Issue #, if available:*
*Description of changes:*
- Add pointers to fev and JumpStart notebook in the README
- Add notebook describing how to deploy Chronos with SageMaker JumpStart
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
