{"author":"David Hou","author_email":"david.hou314@gmail.com","author_time":1711603537,"commit_time":1711603537,"committer":"GitHub","committer_email":"noreply@github.com","hash":"4b95350c4192b940969dbbad378db2ea4c61c88f","message":"fp16 resnet (without expand backwards sum in float, doesn't work) (#3816)\n\n* fp16 resnet\r\n\r\n* cast running mean and var back to default float\r\n\r\n* extra cast\r\n\r\n* check symbolic no overflow\r\n\r\n* add linearizer failure\r\n\r\n* loss scaler after grad contig\r\n\r\n* oops\r\n\r\n* i think this works\r\n\r\n* don't loss scale fp32\r\n\r\n* remove overflow test case\r\n\r\n* remove symbolic bounds check\r\n\r\n* loss scaler should be float\r\n\r\n* temporarily disable padto cuz bug\r\n\r\nshruggie\r\n\r\n* make running stats in batchnorm float32?\r\n\r\n* calculate lars stuff in fp32?\r\n\r\n* oops\r\n\r\n* remove most changes\r\n\r\n* move loss scaler out of optimizer\r\n\r\n* no more FP16 var\r\n\r\n* oops\r\n\r\n---------\r\n\r\nCo-authored-by: chenyu <chenyu@fastmail.com>","parents":["607b4a7d704654b208d89a58d19e54d9cce353c9"],"tree_hash":"da85dc593cbf3911bac4c6291777e829e5d84b53"}