Skip to content

A layer's explicit dtype= policy is not read, so the compute-dtype cast assumes the default #822

Description

@khatchad

The compute-dtype cast added for #821 assumes the default dtype policy, so a layer constructed with an explicit dtype= is reported as if it carried float32.

Reproduction

import numpy as np
import tensorflow as tf


class Wide(tf.keras.layers.Layer):
    def __init__(self):
        super().__init__(dtype="float64")

    def call(self, x):
        assert x.dtype == tf.float64
        return x


Wide()(tf.constant(np.ones((2, 3), dtype=np.float64)))

The assertion holds at run time: a layer whose policy is float64 leaves a float64 input alone. Ariadne reports float32.

Why It Was Left

The cast is right for every layer under the default policy, which is the common case, and the previous behavior was wrong for all of them. A layer with an explicit dtype= was accidentally right before and is wrong now, so this is a narrowing of one case against a correction of many.

Guessing at the policy would be worse than either. Reading it means resolving the dtype keyword through the constructor to Layer.__init__, including the **kwargs forwarding that subclass constructors commonly use, and a wrong read gives a confidently wrong dtype rather than an honest one.

What A Fix Needs

The layer class's construction sites, and the dtype keyword's literal value where one is passed. The recognizer for the construction is the same class-name-suffix scan explicitBuildContractShapes uses to find build callers, so the machinery exists. When no explicit dtype is found the default stands; when one is found and does not resolve to a literal, the honest answer is to leave the caller's type alone rather than assert either dtype.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions