Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions .github/workflows/python-tox.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,6 @@ jobs:
matrix:
os: [
ubuntu-latest,
macos-latest,
windows-latest,
]

steps:
Expand All @@ -35,7 +33,6 @@ jobs:
strategy:
matrix:
python: [
"3.10",
"3.12",
]

Expand Down
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@
.vscode/

# cache files
temp*/
data*/
img*/
temp*/

# Jupyter Notebook Checkpoints
*.ipynb_checkpoints/
Expand Down Expand Up @@ -49,6 +49,7 @@ pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.pytest_cache/
.tox/
.coverage
.coverage.*
Expand All @@ -57,6 +58,7 @@ nosetests.xml
coverage.xml
*,cover
.hypothesis/
.benchmarks/

# Translations
*.mo
Expand Down
41 changes: 41 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,46 @@
# Changelog

## 2026/09/02 - 00 - v0.1.0 - Bump version and Minor Fixes
* Minor fixes to `quantrl.backends.jax`, `quantrl.envs.base` and `quantrl.envs.stochastic` modules.
* Renamed ``seed`` to ``noise_seed`` in `quantrl.envs.base` module.
* Updated ``LearningCurvePlotter`` class in `quantrl.plotters` module.
* Bump version in `docs/source/conf.py`, `quantrl.__init__.py`.
* Updated `README`.

## 2026/07/16 - 00 - v0.0.11 - Add Support for Tuple Spaces
* Added support for tuple spaces to combine discrete and continuious actions in `quantrl.envs.base` module.
* Added option for maximum number of steps for `DiffraxIVPSolver` in `quantrl.solver.jax` module.
* Minor updates to `quantrl.backends.torch` module.

## 2025/08/20 - 00 - v0.0.10 - Code Cleanup
* Minor changes to all modules except `quantrl.solvers.measure`.
* Updated indentations and line lengths of modules and `README`.

## 2025/05/12 - 00 - v0.0.10 - PyTorch GPU Support
* Removed CI builds for MacOS and Windows.
* Changes to `quantrl.backends` package:
* Updated context managers to catch import errors.
* Renamed `jax.JaxBackend` class to `jax.JAXBackend`.
* Minor fixes to CUDA options for `torch.PyTorchBackend` class.
* Minor fixes to `quantrl.envs` package modules.
* Changes to `quantrl.solvers` package:
* Update context managers to catch import errors.
* Minor fixes to `jax` and `numpy` modules.
* Added ``'tsit5'`` option in `pytorch.TorchDiffEqIVPSolver` class.
* Bumped version to `0.0.10` in `docs/source/conf.py` and `quantrl.__init__.py`.
* Added tests for `quantrl.envs.stochastic` module.
* Updated `.gitignore` and `pylintrc`.
* Changes to `pyproject.toml`:
* Removed optional import for PyTorch, which is now installed as a dependency.
* Added `pytest` tests and renamed ``'lint'`` environment to ``'test'`` for `tox`.
* Updated `CONTRIBUTING`, `requirements` and `README`.

## 2025/05/10 - 00 - v0.0.9 - Update Requirements
* Added support for NumPy version `2.0+` with minimum Python version `3.12`.
* Minor fixes to `quantrl.backends.jax` and `quantrl.io` modules.
* Minor changes to `CONTRIBUTING.md` and `pylintrc`.
* Updated `README`, `pyproject.toml` and `requirements`.

## 2024/10/14 - 00 - v0.0.8 - Instantiation and GitHub CI
* Instantiated backends and solvers with different numerical libraries:
* Added `context_manager` modules to `quantrl.backends` and `quantrl.solvers` packages.
Expand Down
23 changes: 19 additions & 4 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,21 +57,36 @@ ROOT_DIR/
├───CODE_OF_CONDUCT.md
├───CONTRIBUTING.md
├───LICENSE
├───MANIFEST.in
├───pylintrc
├───pyproject.toml
├───README.md
├───requirements.txt
└───setup.py
└───requirements_tox.txt
```

### Installing in Editable Mode

To install the package in editable mode, execute the following from *outside* the top-level directory, `ROOT_DIR`, inside which `setup.py` is located:
To install the package in editable mode, execute the following from *inside* the top-level directory, `ROOT_DIR`, inside which `setup.py` is located:

```bash
pip install -e ROOT_DIR
python -m pip install -r requirements.txt
python -m pip install -e .
```

To install the JAX dependencies, use:

```bash
python -m pip install -e .[jax-cpu]
```

for the CPU version, or,

```bash
python -m pip install -e .[jax-gpu]
```

for the GPU version with CUDA 12.

### Building the Documentation

To auto-generate and build the API documentation, navigate to the `ROOT_DIR/docs` folder and execute:
Expand Down
71 changes: 44 additions & 27 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,55 +1,72 @@
# QuantRL: Quantum Control using Reinforcement Learning

![Latest Version](https://img.shields.io/badge/version-0.0.8-red?style=for-the-badge)
![Latest Version](https://img.shields.io/badge/version-0.1.0-red?style=for-the-badge)

> A library of modules to interface deterministic and stochastic quantum models for reinforcement learning.
> A backend-agnostic library of modules to interface
deterministic and stochastic quantum models for reinforcement learning.

### Key Features!

* Quickly interface environments for Reinforcement Learning using Stable-Baselines3.
* Run multiple environments in parallel using vectorized inheritable classes.
* Support for deterministic and stochastic linear environments.
* Live visualization and learning curves.
* Quickly interface environments with any of the three backends:
NumPy, PyTorch and JAX.
* Run multiple RL environments in parallel using
vectorized inheritable classes.
* Evolve deterministic and stochastic environments
with asynchronous saves.
* Visualize evolutions and plot learning curves seamlessly.

### What's New in v0.0.7
### What's New!

* Added support for measurement noise in observations.
* Updated stochastic environment for fast Wiener processes.
* Asynchronous cache-dump to speed up environment evolution.
* Callback to save best mean reward.
* Support for PyTorch (GPU) with ``'tsit'`` solver.
* Support for tuple action spaces.

### What's New in v0.0.6

* Initialize environments with any of the three backends: NumPy, PyTorch and JAX.
* Solve IVPs for the popular libraries `TorchDiffEq` and `Diffrax`.
For a complete list of changes, see [CHANGELOG.md](CHANGELOG.md).

## Installation

[QuantRL](https://github.com/sampreet/quantrl) requires `Python 3.10+`, preferably installed via the [Anaconda distribution](https://www.anaconda.com/download).
The toolbox primarily relies on `gymnasium` (for single environments) and `stable-baselines3` (for vectorized environments).
All of its dependencies can be installed using:
[QuantRL](https://github.com/sampreet/quantrl) requires `Python 3.12+`,
preferably installed via the
[MiniForge distribution](https://conda-forge.org/download/).
It's base dependencies can be installed using:

```bash
conda install "numpy<2.0.0" scipy matplotlib tqdm pillow pandas gymnasium stable-baselines3
python -m pip install numpy scipy matplotlib tqdm rich pillow pandas
```

Additionally, to avail the PyTorch or JAX backends, the latest version of these framework (for both CPU and GPU) should be installed (preferably in different `conda` environments) using in their official documentations: [PyTorch docs](https://pytorch.org/get-started/locally/) and [JAX docs](https://jax.readthedocs.io/en/latest/installation.html).
After successful installation, the corresponding libraries (`torchdiffeq` for PyTorch and `diffrax` for JAX) can be installed using PIP as:
The default backend for the library uses vanilla NumPy and Scipy.
To avail the JAX or PyTorch backends, the latest version
of these framework (CPU or GPU) should be installed
(preferably in different `conda` environments)
using in their official documentations:
[JAX docs](https://jax.readthedocs.io/en/latest/installation.html) and
[PyTorch docs](https://pytorch.org/get-started/locally/).
After successful installation, the corresponding libraries
(`diffrax` for JAX and `torchdiffeq` for PyTorch) can be installed using PIP.

For the CPU versions, use:

```bash
pip install torchdiffeq
python -m pip install torch torchdiffeq jax diffrax
```

or,
For the GPU versions with CUDA 13 support, use:

```bash
pip install jax
pip install diffrax
python -m pip install torch --index-url https://download.pytorch.org/whl/cu126
python -m pip install torchdiffeq "jax[cuda13]" diffrax
```

To install JAX with GPU support, use `jax[cuda12]`.
***Note: JAX-GPU support for Windows and MacOS
is still limited but it runs well in WSL2.***

QuantRL primarily relies on `gymnasium` (for single environments)
and `stable-baselines3` (for vectorized environments).

***Note: JAX-GPU support for Windows and MacOS is still limited but it runs well in WSL2.***
These can be installed using:

```bash
python -m pip install gymnasium stable-baselines3
```

Finally, to install the latest version of `quantrl`, execute:

Expand Down
2 changes: 1 addition & 1 deletion docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
author = 'Sampreet Kalita'

# The full version, including alpha/beta/rc tags
release = '0.0.7'
release = '0.1.0'


# -- General configuration ---------------------------------------------------
Expand Down
8 changes: 4 additions & 4 deletions pylintrc
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,14 @@ disable=too-many-lines,
too-many-public-methods,
too-many-instance-attributes,
too-many-positional-arguments,
line-too-long, # no bounded lines
line-too-long, # catches docstring tables
redefined-builtin, # catches __name__
duplicate-code, # catches __init__
unused-argument, # catches Gym.Env methods
import-outside-toplevel, # for numerical libraries
import-outside-toplevel, # for context managers
not-callable, # catches JAX JIT functions
fixme

unnecessary-lambda, # catches JAX JIT functions
fixme,
[BASIC]
attr-naming-style=any
variable-naming-style=any
Expand Down
24 changes: 11 additions & 13 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[build-system]
requires = ["cython", "setuptools>=61", "wheel"]
requires = ["cython", "setuptools", "wheel"]
build-backend = "setuptools.build_meta"

[project]
Expand All @@ -20,14 +20,17 @@ keywords = [
]
readme = "README.md"
license = {file = "LICENSE"}
requires-python = ">=3.10"
requires-python = ">=3.12"
dependencies = [
"numpy<2.0.0",
"numpy",
"scipy",
"matplotlib",
"tqdm",
"rich",
"pillow",
"pandas",
"torch",
"torchdiffeq",
"gymnasium",
"stable-baselines3",
]
Expand All @@ -49,12 +52,6 @@ jax-gpu = [
"jax[cuda12]",
"diffrax",
]
torch = [
"torch",
"torchvision",
"torchaudio",
"torchdiffeq",
]

[project.urls]
Homepage = "https://github.com/sampreet/quantrl"
Expand All @@ -76,12 +73,13 @@ requires =
tox>=4
virtualenv>=20
env_list =
lint
test

[testenv:lint]
description = run pylint under {base_python}
[testenv:test]
description = run tests under {base_python}
deps =
-r requirements_tox.txt
commands =
pylint quantrl
pylint quantrl tests
pytest tests --cov=quantrl --cov-report=term-missing
"""
2 changes: 1 addition & 1 deletion quantrl/__init__.py
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
"""Module to initialize QuantRL."""
__version__ = "0.0.8"
__version__ = "0.1.0"
Loading
Loading