bpf: htab: Reduce memory use of hash maps

[kernel-patches-daemon-bpf]

Pull request for series with
subject: bpf: htab: Reduce memory use of hash maps
version: 4
url: https://patchwork.kernel.org/project/netdevbpf/list/?series=1145038

[kernel-patches-daemon-bpf]

Upstream branch: 3a59f11
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=1145038
version: 4

[kernel-patches-daemon-bpf]

Upstream branch: 3a59f11
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=1145038
version: 4

[kernel-patches-review-bot]

commit 5ac3acc5b5449639a3a99a19b19bd9da2f4d6ca0
Author: T.J. Mercier <tjmercier@google.com>

bpf: htab: Reduce elem_size by 8 bytes for small key sizes

This patch eliminates the cached hash field for standard and PCPU hash
maps with small keys, reducing elem_size by 8 bytes. It introduces
htab_has_hash() and updates the lookup paths to support atomic key
comparisons when the hash is omitted.

> diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c
> index fed4b1db8b78b..720766121ea2b 100644
> --- a/kernel/bpf/hashtab.c
> +++ b/kernel/bpf/hashtab.c

[ ... ]

> diff --git a/tools/testing/selftests/bpf/progs/map_ptr_kern.c b/tools/testing/selftests/bpf/progs/map_ptr_kern.c
> index 373c8d17ea557..6bd4cb68c20c1 100644
> --- a/tools/testing/selftests/bpf/progs/map_ptr_kern.c
> +++ b/tools/testing/selftests/bpf/progs/map_ptr_kern.c
> @@ -114,7 +114,7 @@ static inline int check_hash(void)
>  	VERIFY(check_default_noinline(&hash->map, map));
>
>  	VERIFY(hash->n_buckets == MAX_ENTRIES);
> -	VERIFY(hash->elem_size == 64);
> +	VERIFY(hash->elem_size == 32);
>
>  	VERIFY(hash->count.counter == 0);
>  	VERIFY(bpf_map_sum_elem_count(map) == 0);

Should this expectation change have been split across two commits for
bisectability? The preceding commit 1066507e33e67 dropped the
freelist/lru union from struct htab_elem, which moves elem_size to 40,
but that commit leaves the selftest checking for 64. This means the
map_ptr selftest fails at 1066507e33e67 and only passes again here
where elem_size becomes 32.

---

AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

In-Reply-To-Subject: bpf: htab: Reduce elem_size by 8 bytes for small key sizes
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31651309438

[kernel-patches-review-bot]

commit 1066507e33e671656204f6a3430e3a9d3e1f8fd6
Author: T.J. Mercier <tjmercier@google.com>

bpf: htab: Split htab_elem_lru and htab_elem_pcpu off of htab_elem

This patch eliminates per-element memory overhead by splitting htab_elem
into dedicated structures for each map type (htab_elem, htab_elem_pcpu,
htab_elem_lru) and adding a runtime key_offset field to struct bpf_htab.

> @@ -476,7 +522,7 @@ static void htab_mem_dtor(void *obj, void *ctx)
>  	if (IS_ERR_OR_NULL(hrec->record))
>  		return;
>
> -	map_value = htab_elem_value(elem, hrec->key_size);
> +	map_value = (void *)elem + sizeof(struct htab_elem) + round_up(hrec->key_size, 8);
>  	bpf_obj_free_fields(hrec->record, map_value);
>  }

This isn't a bug, but would it be worth storing the element header offset
in struct htab_btf_record (next to key_size) and keeping htab_mem_dtor on
the shared htab_elem_value() path, rather than open-coding
sizeof(struct htab_elem) here? Everywhere else in the file the header
size is now the per-map htab->key_offset, reached through
htab_elem_key()/htab_elem_value(). The hardcoded form happens to be
correct today because htab_mem_dtor is only installed for non-preallocated,
non-percpu, non-LRU BPF_MAP_TYPE_HASH maps (LRU requires prealloc, and
HASH_OF_MAPS uses map_check_no_btf), which is exactly the case where
key_offset == sizeof(struct htab_elem). The invariant is invisible at the
call site and there is no comment or assertion tying the two together, so
the open-coded copy silently diverges from the accessor if another element
layout ever registers this dtor.

> @@ -869,17 +929,17 @@ static bool htab_lru_map_delete_node(void *arg, struct bpf_lru_node *node)
>  	struct bucket *b;
>  	int ret;
>
> -	tgt_l = container_of(node, struct htab_elem, lru_node);
> -	b = __select_bucket(htab, tgt_l->hash);
> +	tgt_l = (struct htab_elem *)container_of(node, struct htab_elem_lru, lru_node);
> +	b = __select_bucket(htab, htab_elem_hash(htab, tgt_l));

This isn't a bug, but the commit adds htab_elem_lru_node() for the
htab_elem to bpf_lru_node direction, while the inverse conversion stays
open-coded and is now duplicated verbatim at two sites (line 358 in
prealloc_lru_pop() and line 932 here), each needing an explicit
(struct htab_elem *) cast to silence the incompatible-pointer warning.
Would a small counterpart helper, e.g. htab_elem_from_lru_node(), keep
both directions symmetric and confine the cast to one place?

Commit message issue:

>     bpf: htab: Split htab_elem_lru and htab_elem_pcpu off of htab_elem
>
>     The htab_elem struct is used as the per-element type for all BPF hash
>     map types and includes bpf_lru_node in a union with a ptr_to_pptr
>     pointer. For standard (non-LRU, non-PCPU) hash maps, the 24 byte union
>     allocated for every element is entirely unused. For non-preallocated
>     PCPU maps, ptr_to_pptr only requires 8 bytes, leaving 16 bytes of unused
>     overhead in the union. For preallocated PCPU maps ptr_to_pptr is unused
>     since elements are freed to the PCPU freelist.
>
>     Eliminate this per-element memory overhead by splitting htab_elem into
>     dedicated structures for each map type:
>        - struct htab_elem: Minimal structure for standard hash maps and
>          preallocated PCPU maps (saves 24 bytes per element).
>        - struct htab_elem_pcpu: Structure for non-preallocated PCPU maps
>          containing ptr_to_pptr (saves 16 bytes per element).
>        - struct htab_elem_lru: Retains struct bpf_lru_node for LRU maps.
>
>     Because element sizes now vary by map type, add key_offset to struct
>     bpf_htab to track the dynamic key offset. Update helper accessors and
>     lookups to compute key and value offsets using htab->key_offset.
>
>     Pointers to struct htab_elem in the existing code (e.g. htab_elem_hash)
>     serve as generic base element pointers. This is possible because
>     htab_elem, htab_elem_pcpu, and htab_elem_lru share a common initial
>     sequence, making pointer casts safe.

This isn't a bug, but could the changelog gain a paragraph on the
map_in_map.c side? The patch also touches kernel/bpf/map_in_map.c and
kernel/bpf/map_in_map.h: it adds bpf_map_type_is_htab(), grows
inner_map_meta to sizeof(struct bpf_htab) via the new
bpf_htab_map_meta_size(), and copies key_offset into the meta map with the
new bpf_htab_map_meta_init(). That part is the non-obvious
correctness-critical consequence of making the key offset a runtime field,
since the verifier's inlined lookups read the offset out of the
inner_map_meta. A reviewer looking at the diff would most benefit from an
explanation of why inner_map_meta must now be sizeof(struct bpf_htab) and
have key_offset copied. Also, the sentence about making pointer casts safe
mentions the common initial sequence but doesn't name which casts rely on
it (htab_elem_lru_node(), htab_elem_get_ptr_to_pptr(),
htab_elem_set_ptr_to_pptr()).

---

AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

In-Reply-To-Subject: bpf: htab: Split htab_elem_lru and htab_elem_pcpu off of htab_elem
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31651309438

[kernel-patches-daemon-bpf]

Forwarding comment 5274302490 via email
In-Reply-To: 20260812231905.2956588-3-tjmercier@google.com
Patch: https://patchwork.kernel.org/project/netdevbpf/patch/20260812231905.2956588-3-tjmercier@google.com/

[kernel-patches-daemon-bpf]

Forwarding comment 5274306118 via email
In-Reply-To: 20260812231905.2956588-2-tjmercier@google.com
Patch: https://patchwork.kernel.org/project/netdevbpf/patch/20260812231905.2956588-2-tjmercier@google.com/

[kernel-patches-daemon-bpf]

Upstream branch: 108d440
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=1145038
version: 4

[kernel-patches-daemon-bpf]

Upstream branch: a88dbe1
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=1145038
version: 4

[kernel-patches-daemon-bpf]

Upstream branch: 7c3e54c
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=1145038
version: 4

[kernel-patches-daemon-bpf]

Upstream branch: 6f03361
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=1145038
version: 4