diff --git a/CHANGELOG.md b/CHANGELOG.md index 79845fe..a7e0fc4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,37 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [2.0.0] - 2025-02-06 + +### Breaking Changes + +- **`get_at` now raises `PathError` by default** for missing paths instead of returning `None` silently. Use the `default` parameter for optional/nullable access: `get_at(data, "path", default=None)` +- **`get_at` and `set_at` parameters are now keyword-only** - The `default` and `create` parameters must be passed as keyword arguments (e.g., `get_at(data, "path", default=None)`, not `get_at(data, "path", None)`) +- **`set_at` parameter change** - The `fill_strategy` parameter has been replaced with a simpler `create` boolean parameter. Replace `fill_strategy=FillStrategy.AUTO` with `create=True` +- **No more sparse lists** - `set_at` no longer allows creating lists with gaps. Lists must be built sequentially (index 0, then 1, then 2, etc.). Attempting to set at index > len(list) raises `PathError` +- **Removed `FillStrategy` enum** - Use `create=True/False` instead +- **Implicit `None` returns from `get_at`** - Now raises `PathError` by default instead of returning `None` silently + +### Added + +- **Introspection module** with new functions for analyzing nested structures: + - `get_depth(data)` - Returns maximum nesting depth + - `count_leaves(data)` - Counts total leaf values + - `get_all_paths(data)` - Returns all paths to leaf values +- **`default` parameter for `get_at`** - Explicit way to handle missing paths: `get_at(data, "path", default="fallback")` +- **`create` parameter for `set_at`** - Simple boolean to control auto-creation of intermediate containers +- **New error codes**: + - `OPERATION_DISABLED` - For operations blocked by configuration (e.g., list deletion without `allow_list_mutation=True`) + - `NON_NAVIGABLE_TYPE` - For attempts to navigate into non-container types (e.g., int, str, set) + +### Changed + +- Improved error messages with more context about what went wrong and how to fix it +- Refactored internal helpers for better maintainability and testability +- Enhanced path validation with clearer error codes + +For detailed migration instructions, see the [Migration Guide](https://ysskrishna.github.io/nestedutils/migration-v1-to-v2/). + ## [1.1.7] ### Added @@ -13,7 +44,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Radio button selection for three example datasets (User Profile, E-commerce Data, API Response) and custom JSON input - Consolidated path validation tests into `test_normalize_path.py` with new test cases for complex keys and None value handling - ### Fixed - Fixed `normalize_path()` converting all list path keys to strings, now preserves integer and other key types. @@ -133,6 +163,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Immutable container protection (tuples cannot be modified) - Safe list deletion (requires explicit `allow_list_mutation=True` flag) +[2.0.0]: https://github.com/ysskrishna/nestedutils/compare/v1.1.7...v2.0.0 [1.1.7]: https://github.com/ysskrishna/nestedutils/compare/v1.1.6...v1.1.7 [1.1.6]: https://github.com/ysskrishna/nestedutils/compare/v1.1.5...v1.1.6 [1.1.5]: https://github.com/ysskrishna/nestedutils/compare/v1.1.4...v1.1.5 diff --git a/README.md b/README.md index 0b0ee36..56f0bae 100644 --- a/README.md +++ b/README.md @@ -37,8 +37,8 @@ user_name = get_at(data, "users.0.profile.name") - **Simple Path Syntax**: Use dot-notation strings (`"a.b.c"`) or lists (`["a", "b", "c"]`) to navigate nested structures - **Mixed Data Types**: Seamlessly work with dictionaries, lists, and tuples (read-only for tuples) - **List Index Support**: Access list elements using numeric indices, including negative indices -- **Auto-creation**: Automatically create missing intermediate containers when setting values -- **Flexible Fill Strategies**: Control how missing containers are created with different fill strategies +- **Auto-creation**: Automatically create missing intermediate containers when setting values (with `create=True`) +- **Introspection**: Analyze nested structures with `get_depth`, `count_leaves`, and `get_all_paths` - **Type Safety**: Comprehensive error handling with descriptive error messages and error codes - **Safety Limits**: Built-in protection against excessive nesting (max depth: 100) and oversized lists (max index: 10,000) - **Zero Dependencies**: Pure Python implementation with no external dependencies @@ -49,6 +49,14 @@ user_name = get_at(data, "users.0.profile.name") - **Configuration Management**: Easily read and modify deeply nested settings in configuration dictionaries. - **Data Transformation**: Rapidly remap data from one complex structure to another using `get_at` and `set_at`. +## Terminology + +| Term | Definition | +|------|------------| +| **Path** | A navigation string or list that specifies a location in nested data (e.g., `"user.profile.name"` or `["user", "profile", "name"]`) | +| **Key** | An individual dictionary key used to access a value (e.g., `"name"`, `"profile"`) | +| **Index** | A numeric position in a list or tuple (e.g., `0`, `-1` for last element) | + ## Installation ```bash @@ -58,16 +66,16 @@ pip install nestedutils ## Quick Start ```python -from nestedutils import get_at, set_at, delete_at, exists_at +from nestedutils import get_at, set_at, delete_at, exists_at, get_depth, count_leaves, get_all_paths # Create a nested structure data = {} # Set values using dot-notation -set_at(data, "user.name", "John") -set_at(data, "user.age", 30) -set_at(data, "user.hobbies.0", "reading") -set_at(data, "user.hobbies.1", "coding") +set_at(data, "user.name", "John", create=True) +set_at(data, "user.age", 30, create=True) +set_at(data, "user.hobbies.0", "reading", create=True) +set_at(data, "user.hobbies.1", "coding", create=True) # Access values name = get_at(data, "user.name") # "John" @@ -84,7 +92,7 @@ delete_at(data, "user.age") ## API Reference -### `get_at(data, path, default=None)` +### `get_at(data, path, *, default=None)` Retrieve a value from a nested data structure. @@ -92,58 +100,67 @@ Retrieve a value from a nested data structure. - `data`: The data structure to navigate (dict, list, tuple, or nested combinations) - `path`: Path to the value (string with dot notation or list of keys/indices) -- `default`: Value to return if path doesn't exist (default: `None`) +- `default`: Value to return if path doesn't exist (keyword-only parameter, default: `None`) + +**Returns:** The value at the path, or `default` if provided and path doesn't exist -**Returns:** The value at the path, or `default` if not found +**Raises:** `PathError` if the path doesn't exist and `default` is not provided + +**Note:** By default, `get_at` raises `PathError` for missing paths. Use the `default` parameter for optional/nullable access. **Examples:** ```python data = {"a": {"b": {"c": 5}}} get_at(data, "a.b.c") # 5 -get_at(data, "a.b.d", default=99) # 99 +get_at(data, "a.b.d") # Raises PathError (path doesn't exist) +get_at(data, "a.b.d", default=99) # 99 (returns default) data = {"items": [{"name": "apple"}, {"name": "banana"}]} get_at(data, "items.1.name") # "banana" get_at(data, "items.-1.name") # "banana" (negative index) ``` -### `set_at(data, path, value, fill_strategy="auto")` +### `set_at(data, path, value, *, create=False)` -Set a value in a nested data structure, creating intermediate containers as needed. +Set a value in a nested data structure, optionally creating intermediate containers as needed. **Parameters:** - `data`: The data structure to modify (must be mutable: dict or list) - `path`: Path where to set the value (string with dot notation or list of keys/indices) - `value`: The value to set -- `fill_strategy`: How to fill missing containers (default: `"auto"`) - - `"auto"`: Intelligently creates `{}` for dict keys, `[]` for list indices, and `None` for sparse list gaps - - `"none"`: Fills missing list items with `None` - - `"dict"`: Always creates dictionaries - - `"list"`: Always creates lists +- `create`: If `True`, automatically creates missing intermediate containers (default: `False`) -**Note:** Positive indices can extend lists (filling gaps as needed), but negative indices can only modify existing elements. +**Note:** +- By default (`create=False`), `set_at` raises `PathError` if any intermediate key is missing +- With `create=True`, missing containers are automatically created: `{}` for dict keys, `[]` for list indices +- Positive indices can append to lists (index == len(list)) but cannot create gaps (index > len(list)) +- Negative indices can only modify existing elements **Examples:** ```python +# create=True - auto-create missing containers data = {} -set_at(data, "user.profile.name", "Alice") +set_at(data, "user.profile.name", "Alice", create=True) # Creates: {"user": {"profile": {"name": "Alice"}}} data = {} -set_at(data, "items.0.name", "Item 1") +set_at(data, "items.0.name", "Item 1", create=True) # Creates: {"items": [{"name": "Item 1"}]} +# Sequential list appending (no gaps allowed) data = {} -set_at(data, "items.5", "Item 6", fill_strategy="none") -# Creates: {"items": [None, None, None, None, None, "Item 6"]} +set_at(data, "items.0", "first", create=True) # Creates list with first item +set_at(data, "items.1", "second", create=True) # Appends second item +# Creates: {"items": ["first", "second"]} +# Sparse lists are NOT allowed - this raises PathError data = [1, 2, 3] -set_at(data, "5", 99) # Extends list with None gaps -# Creates: [1, 2, 3, None, None, 99] +set_at(data, "5", 99, create=True) # Raises PathError: cannot create gap +# Negative indices - modify existing only data = [1, 2, 3] set_at(data, "-1", 100) # Updates existing last element # Creates: [1, 2, 100] @@ -185,6 +202,8 @@ Delete a value from a nested data structure. - `path`: Path to the value to delete - `allow_list_mutation`: If `True`, allows deletion from lists (default: `False`) +**Note:** List deletion is disabled by default to prevent accidental index shifting that could break subsequent code. When you delete an element from a list, all following indices shift down, which can cause unexpected behavior if other parts of your code reference those indices. + **Returns:** The deleted value **Raises:** `PathError` if the path doesn't exist or deletion is not allowed @@ -200,6 +219,79 @@ delete_at(data, "items.1", allow_list_mutation=True) # Returns 2 # data becomes {"items": [1, 3]} ``` +### `get_depth(data)` + +Get the maximum nesting depth of a data structure. + +**Parameters:** + +- `data`: Any nested structure (dict, list, tuple, or primitive) + +**Returns:** Integer depth. Primitives return 0, empty containers return 1. + +**Note:** Only dict, list, and tuple are traversed. Other container types (set, frozenset, etc.) are treated as leaf values. + +**Examples:** + +```python +get_depth(42) # 0 (primitive) +get_depth({}) # 1 (empty container) +get_depth({"a": 1}) # 1 (flat dict) +get_depth({"a": {"b": 1}}) # 2 (nested) +get_depth({"a": {"b": {"c": 1}}}) # 3 (deeper nesting) +get_depth([1, [2, [3]]]) # 3 (nested lists) +``` + +### `count_leaves(data)` + +Count the total number of leaf values (non-container values) in a nested structure. + +**Parameters:** + +- `data`: Any nested structure + +**Returns:** Integer count of leaf values. Empty containers return 0. + +**Note:** Only dict, list, and tuple are traversed. Other container types (set, frozenset, etc.) count as a single leaf. + +**Examples:** + +```python +count_leaves(42) # 1 (primitive is a leaf) +count_leaves({}) # 0 (empty container) +count_leaves({"a": 1, "b": 2}) # 2 (two leaf values) +count_leaves({"a": {"b": 1, "c": 2}}) # 2 (nested, still 2 leaves) +count_leaves([1, 2, [3, 4]]) # 4 (four leaf values) +``` + +### `get_all_paths(data)` + +Get all paths to leaf values in a nested structure. + +**Parameters:** + +- `data`: Any nested structure + +**Returns:** List of paths, where each path is a list of keys (strings) and indices (integers). + +**Note:** Only dict, list, and tuple are traversed. Other container types are treated as leaves. + +**Examples:** + +```python +get_all_paths({"a": 1, "b": 2}) +# [["a"], ["b"]] + +get_all_paths({"a": {"b": 1, "c": 2}}) +# [["a", "b"], ["a", "c"]] + +get_all_paths({"users": [{"name": "Alice"}, {"name": "Bob"}]}) +# [["users", 0, "name"], ["users", 1, "name"]] + +get_all_paths({}) # [] (no leaves) +get_all_paths(42) # [[]] (primitive has empty path) +``` + ## Error Handling The library uses `PathError` exceptions with error codes for different failure scenarios: @@ -216,12 +308,15 @@ except PathError as e: **Error Codes:** -- `INVALID_PATH`: Invalid path format or type -- `INVALID_INDEX`: Invalid list index -- `MISSING_KEY`: Key doesn't exist in dictionary -- `EMPTY_PATH`: Path is empty -- `IMMUTABLE_CONTAINER`: Attempted to modify a tuple -- `INVALID_FILL_STRATEGY`: Invalid fill strategy value +| Error Code | Description | +|------------|-------------| +| `INVALID_PATH` | Invalid path format or type | +| `INVALID_INDEX` | Invalid list index | +| `MISSING_KEY` | Key doesn't exist in dictionary | +| `EMPTY_PATH` | Path is empty | +| `IMMUTABLE_CONTAINER` | Attempted to modify a tuple | +| `NON_NAVIGABLE_TYPE` | Attempted to navigate into a non-container type | +| `OPERATION_DISABLED` | Operation is disabled by configuration (e.g., list deletion without `allow_list_mutation=True`) | ## Advanced Usage @@ -231,8 +326,8 @@ List paths are useful when keys contain dots: ```python data = {} -set_at(data, ["user.name", "first"], "John") -set_at(data, ["user.name", "last"], "Doe") +set_at(data, ["user.name", "first"], "John", create=True) +set_at(data, ["user.name", "last"], "Doe", create=True) # Creates: {"user.name": {"first": "John", "last": "Doe"}} ``` @@ -272,19 +367,17 @@ set_at(data, "a.b.c", 10) The library includes built-in safety limits to prevent excessive resource usage: -- **Maximum Path Depth**: 100 levels (prevents deeply nested paths that could cause stack issues) -- **Maximum List Index**: 10,000 (prevents creating extremely large sparse lists) +| Limit | Value | Description | +|-------|-------|-------------| +| **Maximum Path Depth** | 100 levels | Prevents deeply nested paths that could cause stack issues | +| **Maximum List Index** | 10,000 | Prevents creating extremely large sparse lists | These limits help protect against accidental memory exhaustion or performance issues. If you hit these limits, you'll receive a `PathError` with a clear message. -## Links +## Migration from v1.x to v2.0 -- **PyPI**: [pypi.org/project/nestedutils](https://pypi.org/project/nestedutils/) -- **Documentation**: [ysskrishna.github.io/nestedutils](https://ysskrishna.github.io/nestedutils/) -- **Interactive Demo**: [ysskrishna.github.io/nestedutils/demo/](https://ysskrishna.github.io/nestedutils/demo/) -- **Repository**: [github.com/ysskrishna/nestedutils.git](https://github.com/ysskrishna/nestedutils.git) -- **Issues**: [github.com/ysskrishna/nestedutils/issues](https://github.com/ysskrishna/nestedutils/issues) +Version 2.0 introduces breaking changes to make the library safer and more predictable. If you're upgrading from v1.x, please see the [Migration Guide](https://ysskrishna.github.io/nestedutils/migration-v1-to-v2/) for detailed upgrade instructions. ## Contributing @@ -292,21 +385,25 @@ Contributions are welcome! Please read our [Contributing Guide](https://github.c ## Support -If you find this library useful, please consider: +If you find this library helpful: -- ⭐ **Starring** the repository on GitHub to help others discover it. -- 💖 **Sponsoring** to support ongoing maintenance and development. - -[Become a Sponsor on GitHub](https://github.com/sponsors/ysskrishna) | [Support on Patreon](https://patreon.com/ysskrishna) +- ⭐ Star the repository +- 🐛 Report issues +- 🔀 Submit pull requests +- 💝 [Sponsor on GitHub](https://github.com/sponsors/ysskrishna) ## License -MIT License - see [LICENSE](https://github.com/ysskrishna/nestedutils/blob/main/LICENSE) file for details. - +MIT © [Y. Siva Sai Krishna](https://github.com/ysskrishna) - see [LICENSE](https://github.com/ysskrishna/nestedutils/blob/main/LICENSE) file for details. -## Author -**Y. Siva Sai Krishna** +--- -- GitHub: [@ysskrishna](https://github.com/ysskrishna) -- LinkedIn: [ysskrishna](https://linkedin.com/in/ysskrishna) +

+ Author's GitHub • + Author's LinkedIn • + Report Issues • + Package on PyPI • + Package Documentation • + Package Demo +

diff --git a/docs/api-reference.md b/docs/api-reference.md index e7bccda..42a0da0 100644 --- a/docs/api-reference.md +++ b/docs/api-reference.md @@ -1,7 +1,7 @@ --- title: API Reference -description: "Complete API reference for nestedutils. Documentation for get_at, set_at, delete_at, and exists_at functions with parameters, return values, examples, and error handling." +description: "Complete API reference for nestedutils. Documentation for get_at, set_at, delete_at, exists_at, get_depth, count_leaves, and get_all_paths functions with parameters, return values, examples, and error handling." keywords: - nestedutils API @@ -10,6 +10,9 @@ keywords: - set_at - delete_at - exists_at + - get_depth + - count_leaves + - get_all_paths - function documentation - nested data functions - Python API @@ -31,6 +34,16 @@ keywords: - exists_at heading_level: 3 +::: nestedutils.introspection + options: + show_root_heading: true + show_root_toc_entry: false + members: + - get_depth + - count_leaves + - get_all_paths + heading_level: 3 + ::: nestedutils.constants options: show_root_heading: true @@ -48,7 +61,6 @@ keywords: heading_level: 3 members: - PathErrorCode - - FillStrategy members_order: source show_if_no_docstring: true diff --git a/docs/demo.md b/docs/demo.md index 751b0ee..2180681 100644 --- a/docs/demo.md +++ b/docs/demo.md @@ -106,15 +106,10 @@ Try the `nestedutils` library directly in your browser! This page uses [Pyodide] placeholder="Value to set" style="flex: 1; min-width: 150px; padding: 8px; border: 1px solid #ccc; border-radius: 4px;" /> - +