Code Cleanup

CHANGELOG.md

@@ -5,6 +5,37 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [2.0.0] - 2025-02-06
+
+### Breaking Changes
+
+- **`get_at` now raises `PathError` by default** for missing paths instead of returning `None` silently. Use the `default` parameter for optional/nullable access: `get_at(data, "path", default=None)`
+- **`get_at` and `set_at` parameters are now keyword-only** - The `default` and `create` parameters must be passed as keyword arguments (e.g., `get_at(data, "path", default=None)`, not `get_at(data, "path", None)`)
+- **`set_at` parameter change** - The `fill_strategy` parameter has been replaced with a simpler `create` boolean parameter. Replace `fill_strategy=FillStrategy.AUTO` with `create=True`
+- **No more sparse lists** - `set_at` no longer allows creating lists with gaps. Lists must be built sequentially (index 0, then 1, then 2, etc.). Attempting to set at index > len(list) raises `PathError`
+- **Removed `FillStrategy` enum** - Use `create=True/False` instead
+- **Implicit `None` returns from `get_at`** - Now raises `PathError` by default instead of returning `None` silently
+
+### Added
+
+- **Introspection module** with new functions for analyzing nested structures:
+  - `get_depth(data)` - Returns maximum nesting depth
+  - `count_leaves(data)` - Counts total leaf values
+  - `get_all_paths(data)` - Returns all paths to leaf values
+- **`default` parameter for `get_at`** - Explicit way to handle missing paths: `get_at(data, "path", default="fallback")`
+- **`create` parameter for `set_at`** - Simple boolean to control auto-creation of intermediate containers
+- **New error codes**:
+  - `OPERATION_DISABLED` - For operations blocked by configuration (e.g., list deletion without `allow_list_mutation=True`)
+  - `NON_NAVIGABLE_TYPE` - For attempts to navigate into non-container types (e.g., int, str, set)
+
+### Changed
+
+- Improved error messages with more context about what went wrong and how to fix it
+- Refactored internal helpers for better maintainability and testability
+- Enhanced path validation with clearer error codes
+
+For detailed migration instructions, see the [Migration Guide](https://ysskrishna.github.io/nestedutils/migration-v1-to-v2/).
+
 ## [1.1.7]
 
 ### Added
@@ -13,7 +44,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Radio button selection for three example datasets (User Profile, E-commerce Data, API Response) and custom JSON input
 - Consolidated path validation tests into `test_normalize_path.py` with new test cases for complex keys and None value handling
 
-
 ### Fixed
 
 - Fixed `normalize_path()` converting all list path keys to strings, now preserves integer and other key types.
@@ -133,6 +163,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Immutable container protection (tuples cannot be modified)
 - Safe list deletion (requires explicit `allow_list_mutation=True` flag)
 
+[2.0.0]: https://github.com/ysskrishna/nestedutils/compare/v1.1.7...v2.0.0
 [1.1.7]: https://github.com/ysskrishna/nestedutils/compare/v1.1.6...v1.1.7
 [1.1.6]: https://github.com/ysskrishna/nestedutils/compare/v1.1.5...v1.1.6
 [1.1.5]: https://github.com/ysskrishna/nestedutils/compare/v1.1.4...v1.1.5

README.md

@@ -37,8 +37,8 @@ user_name = get_at(data, "users.0.profile.name")
 - **Simple Path Syntax**: Use dot-notation strings (`"a.b.c"`) or lists (`["a", "b", "c"]`) to navigate nested structures
 - **Mixed Data Types**: Seamlessly work with dictionaries, lists, and tuples (read-only for tuples)
 - **List Index Support**: Access list elements using numeric indices, including negative indices
-- **Auto-creation**: Automatically create missing intermediate containers when setting values
-- **Flexible Fill Strategies**: Control how missing containers are created with different fill strategies
+- **Auto-creation**: Automatically create missing intermediate containers when setting values (with `create=True`)
+- **Introspection**: Analyze nested structures with `get_depth`, `count_leaves`, and `get_all_paths`
 - **Type Safety**: Comprehensive error handling with descriptive error messages and error codes
 - **Safety Limits**: Built-in protection against excessive nesting (max depth: 100) and oversized lists (max index: 10,000)
 - **Zero Dependencies**: Pure Python implementation with no external dependencies
@@ -49,6 +49,14 @@ user_name = get_at(data, "users.0.profile.name")
 - **Configuration Management**: Easily read and modify deeply nested settings in configuration dictionaries.
 - **Data Transformation**: Rapidly remap data from one complex structure to another using `get_at` and `set_at`.
 
+## Terminology
+
+| Term | Definition |
+|------|------------|
+| **Path** | A navigation string or list that specifies a location in nested data (e.g., `"user.profile.name"` or `["user", "profile", "name"]`) |
+| **Key** | An individual dictionary key used to access a value (e.g., `"name"`, `"profile"`) |
+| **Index** | A numeric position in a list or tuple (e.g., `0`, `-1` for last element) |
+
 ## Installation
 
 ```bash
@@ -58,16 +66,16 @@ pip install nestedutils
 ## Quick Start
 
 ```python
-from nestedutils import get_at, set_at, delete_at, exists_at
+from nestedutils import get_at, set_at, delete_at, exists_at, get_depth, count_leaves, get_all_paths
 
 # Create a nested structure
 data = {}
 
 # Set values using dot-notation
-set_at(data, "user.name", "John")
-set_at(data, "user.age", 30)
-set_at(data, "user.hobbies.0", "reading")
-set_at(data, "user.hobbies.1", "coding")
+set_at(data, "user.name", "John", create=True)
+set_at(data, "user.age", 30, create=True)
+set_at(data, "user.hobbies.0", "reading", create=True)
+set_at(data, "user.hobbies.1", "coding", create=True)
 
 # Access values
 name = get_at(data, "user.name")  # "John"
@@ -84,66 +92,75 @@ delete_at(data, "user.age")
 
 ## API Reference
 
-### `get_at(data, path, default=None)`
+### `get_at(data, path, *, default=None)`
 
 Retrieve a value from a nested data structure.
 
 **Parameters:**
 
 - `data`: The data structure to navigate (dict, list, tuple, or nested combinations)
 - `path`: Path to the value (string with dot notation or list of keys/indices)
-- `default`: Value to return if path doesn't exist (default: `None`)
+- `default`: Value to return if path doesn't exist (keyword-only parameter, default: `None`)
+
+**Returns:** The value at the path, or `default` if provided and path doesn't exist
 
-**Returns:** The value at the path, or `default` if not found
+**Raises:** `PathError` if the path doesn't exist and `default` is not provided
+
+**Note:** By default, `get_at` raises `PathError` for missing paths. Use the `default` parameter for optional/nullable access.
 
 **Examples:**
 
 ```python
 data = {"a": {"b": {"c": 5}}}
 get_at(data, "a.b.c")  # 5
-get_at(data, "a.b.d", default=99)  # 99
+get_at(data, "a.b.d")  # Raises PathError (path doesn't exist)
+get_at(data, "a.b.d", default=99)  # 99 (returns default)
 
 data = {"items": [{"name": "apple"}, {"name": "banana"}]}
 get_at(data, "items.1.name")  # "banana"
 get_at(data, "items.-1.name")  # "banana" (negative index)
 ```
 
-### `set_at(data, path, value, fill_strategy="auto")`
+### `set_at(data, path, value, *, create=False)`
 
-Set a value in a nested data structure, creating intermediate containers as needed.
+Set a value in a nested data structure, optionally creating intermediate containers as needed.
 
 **Parameters:**
 
 - `data`: The data structure to modify (must be mutable: dict or list)
 - `path`: Path where to set the value (string with dot notation or list of keys/indices)
 - `value`: The value to set
-- `fill_strategy`: How to fill missing containers (default: `"auto"`)
-  - `"auto"`: Intelligently creates `{}` for dict keys, `[]` for list indices, and `None` for sparse list gaps
-  - `"none"`: Fills missing list items with `None`
-  - `"dict"`: Always creates dictionaries
-  - `"list"`: Always creates lists
+- `create`: If `True`, automatically creates missing intermediate containers (default: `False`)
 
-**Note:** Positive indices can extend lists (filling gaps as needed), but negative indices can only modify existing elements.
+**Note:** 
+- By default (`create=False`), `set_at` raises `PathError` if any intermediate key is missing
+- With `create=True`, missing containers are automatically created: `{}` for dict keys, `[]` for list indices
+- Positive indices can append to lists (index == len(list)) but cannot create gaps (index > len(list))
+- Negative indices can only modify existing elements
 
 **Examples:**
 
 ```python
+# create=True - auto-create missing containers
 data = {}
-set_at(data, "user.profile.name", "Alice")
+set_at(data, "user.profile.name", "Alice", create=True)
 # Creates: {"user": {"profile": {"name": "Alice"}}}
 
 data = {}
-set_at(data, "items.0.name", "Item 1")
+set_at(data, "items.0.name", "Item 1", create=True)
 # Creates: {"items": [{"name": "Item 1"}]}
 
+# Sequential list appending (no gaps allowed)
 data = {}
-set_at(data, "items.5", "Item 6", fill_strategy="none")
-# Creates: {"items": [None, None, None, None, None, "Item 6"]}
+set_at(data, "items.0", "first", create=True)  # Creates list with first item
+set_at(data, "items.1", "second", create=True)  # Appends second item
+# Creates: {"items": ["first", "second"]}
 
+# Sparse lists are NOT allowed - this raises PathError
 data = [1, 2, 3]
-set_at(data, "5", 99)  # Extends list with None gaps
-# Creates: [1, 2, 3, None, None, 99]
+set_at(data, "5", 99, create=True)  # Raises PathError: cannot create gap
 
+# Negative indices - modify existing only
 data = [1, 2, 3]
 set_at(data, "-1", 100)  # Updates existing last element
 # Creates: [1, 2, 100]
@@ -185,6 +202,8 @@ Delete a value from a nested data structure.
 - `path`: Path to the value to delete
 - `allow_list_mutation`: If `True`, allows deletion from lists (default: `False`)
 
+**Note:** List deletion is disabled by default to prevent accidental index shifting that could break subsequent code. When you delete an element from a list, all following indices shift down, which can cause unexpected behavior if other parts of your code reference those indices.
+
 **Returns:** The deleted value
 
 **Raises:** `PathError` if the path doesn't exist or deletion is not allowed
@@ -200,6 +219,79 @@ delete_at(data, "items.1", allow_list_mutation=True)  # Returns 2
 # data becomes {"items": [1, 3]}
 ```
 
+### `get_depth(data)`
+
+Get the maximum nesting depth of a data structure.
+
+**Parameters:**
+
+- `data`: Any nested structure (dict, list, tuple, or primitive)
+
+**Returns:** Integer depth. Primitives return 0, empty containers return 1.
+
+**Note:** Only dict, list, and tuple are traversed. Other container types (set, frozenset, etc.) are treated as leaf values.
+
+**Examples:**
+
+```python
+get_depth(42)                          # 0 (primitive)
+get_depth({})                          # 1 (empty container)
+get_depth({"a": 1})                    # 1 (flat dict)
+get_depth({"a": {"b": 1}})             # 2 (nested)
+get_depth({"a": {"b": {"c": 1}}})      # 3 (deeper nesting)
+get_depth([1, [2, [3]]])               # 3 (nested lists)
+```
+
+### `count_leaves(data)`
+
+Count the total number of leaf values (non-container values) in a nested structure.
+
+**Parameters:**
+
+- `data`: Any nested structure
+
+**Returns:** Integer count of leaf values. Empty containers return 0.
+
+**Note:** Only dict, list, and tuple are traversed. Other container types (set, frozenset, etc.) count as a single leaf.
+
+**Examples:**
+
+```python
+count_leaves(42)                       # 1 (primitive is a leaf)
+count_leaves({})                       # 0 (empty container)
+count_leaves({"a": 1, "b": 2})         # 2 (two leaf values)
+count_leaves({"a": {"b": 1, "c": 2}})  # 2 (nested, still 2 leaves)
+count_leaves([1, 2, [3, 4]])           # 4 (four leaf values)
+```
+
+### `get_all_paths(data)`
+
+Get all paths to leaf values in a nested structure.
+
+**Parameters:**
+
+- `data`: Any nested structure
+
+**Returns:** List of paths, where each path is a list of keys (strings) and indices (integers).
+
+**Note:** Only dict, list, and tuple are traversed. Other container types are treated as leaves.
+
+**Examples:**
+
+```python
+get_all_paths({"a": 1, "b": 2})
+# [["a"], ["b"]]
+
+get_all_paths({"a": {"b": 1, "c": 2}})
+# [["a", "b"], ["a", "c"]]
+
+get_all_paths({"users": [{"name": "Alice"}, {"name": "Bob"}]})
+# [["users", 0, "name"], ["users", 1, "name"]]
+
+get_all_paths({})                      # [] (no leaves)
+get_all_paths(42)                      # [[]] (primitive has empty path)
+```
+
 ## Error Handling
 
 The library uses `PathError` exceptions with error codes for different failure scenarios:
@@ -216,12 +308,15 @@ except PathError as e:
 
 **Error Codes:**
 
-- `INVALID_PATH`: Invalid path format or type
-- `INVALID_INDEX`: Invalid list index
-- `MISSING_KEY`: Key doesn't exist in dictionary
-- `EMPTY_PATH`: Path is empty
-- `IMMUTABLE_CONTAINER`: Attempted to modify a tuple
-- `INVALID_FILL_STRATEGY`: Invalid fill strategy value
+| Error Code | Description |
+|------------|-------------|
+| `INVALID_PATH` | Invalid path format or type |
+| `INVALID_INDEX` | Invalid list index |
+| `MISSING_KEY` | Key doesn't exist in dictionary |
+| `EMPTY_PATH` | Path is empty |
+| `IMMUTABLE_CONTAINER` | Attempted to modify a tuple |
+| `NON_NAVIGABLE_TYPE` | Attempted to navigate into a non-container type |
+| `OPERATION_DISABLED` | Operation is disabled by configuration (e.g., list deletion without `allow_list_mutation=True`) |
 
 ## Advanced Usage
 
@@ -231,8 +326,8 @@ List paths are useful when keys contain dots:
 
 ```python
 data = {}
-set_at(data, ["user.name", "first"], "John")
-set_at(data, ["user.name", "last"], "Doe")
+set_at(data, ["user.name", "first"], "John", create=True)
+set_at(data, ["user.name", "last"], "Doe", create=True)
 # Creates: {"user.name": {"first": "John", "last": "Doe"}}
 ```
 
@@ -272,41 +367,43 @@ set_at(data, "a.b.c", 10)
 
 The library includes built-in safety limits to prevent excessive resource usage:
 
-- **Maximum Path Depth**: 100 levels (prevents deeply nested paths that could cause stack issues)
-- **Maximum List Index**: 10,000 (prevents creating extremely large sparse lists)
+| Limit | Value | Description |
+|-------|-------|-------------|
+| **Maximum Path Depth** | 100 levels | Prevents deeply nested paths that could cause stack issues |
+| **Maximum List Index** | 10,000 | Prevents creating extremely large sparse lists |
 
 These limits help protect against accidental memory exhaustion or performance issues. If you hit these limits, you'll receive a `PathError` with a clear message.
 
 
-## Links
+## Migration from v1.x to v2.0
 
-- **PyPI**: [pypi.org/project/nestedutils](https://pypi.org/project/nestedutils/)
-- **Documentation**: [ysskrishna.github.io/nestedutils](https://ysskrishna.github.io/nestedutils/)
-- **Interactive Demo**: [ysskrishna.github.io/nestedutils/demo/](https://ysskrishna.github.io/nestedutils/demo/)
-- **Repository**: [github.com/ysskrishna/nestedutils.git](https://github.com/ysskrishna/nestedutils.git)
-- **Issues**: [github.com/ysskrishna/nestedutils/issues](https://github.com/ysskrishna/nestedutils/issues)
+Version 2.0 introduces breaking changes to make the library safer and more predictable. If you're upgrading from v1.x, please see the [Migration Guide](https://ysskrishna.github.io/nestedutils/migration-v1-to-v2/) for detailed upgrade instructions.
 
 ## Contributing
 
 Contributions are welcome! Please read our [Contributing Guide](https://github.com/ysskrishna/nestedutils/blob/main/CONTRIBUTING.md) for details on our code of conduct, development setup, and the process for submitting pull requests.
 
 ## Support
 
-If you find this library useful, please consider:
+If you find this library helpful:
 
-- ⭐ **Starring** the repository on GitHub to help others discover it.
-- 💖 **Sponsoring** to support ongoing maintenance and development.
-
-[Become a Sponsor on GitHub](https://github.com/sponsors/ysskrishna) | [Support on Patreon](https://patreon.com/ysskrishna)
+- ⭐ Star the repository
+- 🐛 Report issues
+- 🔀 Submit pull requests
+- 💝 [Sponsor on GitHub](https://github.com/sponsors/ysskrishna)
 
 ## License
 
-MIT License - see [LICENSE](https://github.com/ysskrishna/nestedutils/blob/main/LICENSE) file for details.
-
+MIT © [Y. Siva Sai Krishna](https://github.com/ysskrishna) - see [LICENSE](https://github.com/ysskrishna/nestedutils/blob/main/LICENSE) file for details.
 
-## Author
 
-**Y. Siva Sai Krishna**
+---
 
-- GitHub: [@ysskrishna](https://github.com/ysskrishna)
-- LinkedIn: [ysskrishna](https://linkedin.com/in/ysskrishna)
+<p align="left">
+  <a href="https://github.com/ysskrishna">Author's GitHub</a> •
+  <a href="https://linkedin.com/in/ysskrishna">Author's LinkedIn</a> •
+  <a href="https://github.com/ysskrishna/nestedutils/issues">Report Issues</a> •
+  <a href="https://pypi.org/project/nestedutils/">Package on PyPI</a> •
+  <a href="https://ysskrishna.github.io/nestedutils/">Package Documentation</a> •
+  <a href="https://ysskrishna.github.io/nestedutils/demo/">Package Demo</a>
+</p>