{"author":"David Hou","author_email":"david.hou314@gmail.com","author_time":1709265127,"commit_time":1709265127,"committer":"GitHub","committer_email":"noreply@github.com","hash":"e5385eecfc377dae48086e77a095839582535ea4","message":"UnsyncedBatchNorm with synced trainable weights for hlb cifar (#3472)\n\n* UnsyncedBatchNorm with synced trainable weights for hlb cifar\r\n\r\n* multitensor reshape tests\r\n\r\n* test mlb assign change axis\r\n\r\n* E501\r\n\r\n* argfix axis\r\n\r\n* don't import batchnorm from hlb_cifar in test_multitensor\r\n\r\n* pass num_devices to UnsyncedBatchNorm in test, allow UnsyncedBatchNorm to be used with LB\r\n\r\n* add backprop test for UnsyncedBatchNorm\r\n\r\n* break out MLB assign and reshape changes\r\n\r\n* manually shard running mean and running var\r\n\r\n* don't shard unless syncbn=0\r\n\r\n* replace nn.BatchNorm2d with UnsyncedBatchNorm\r\n\r\n* don't increment num_batches_tracked if not tracking running stats\r\n\r\n* update tests\r\n\r\n* oops\r\n\r\n* Revert \"oops\"\r\n\r\nThis reverts commit 5e8a67a535abea2ff288b1b804a9aa95eba40732.\r\n\r\n* Revert \"update tests\"\r\n\r\nThis reverts commit 7ebf65d89ace1d3a32c3b28ee323ddee253262d6.\r\n\r\n* Revert \"don't increment num_batches_tracked if not tracking running stats\"\r\n\r\nThis reverts commit 78de0ea9ee8cbd65dce28bd4abcc131c98451aa2.\r\n\r\n* Revert \"replace nn.BatchNorm2d with UnsyncedBatchNorm\"\r\n\r\nThis reverts commit d03da53da70f009338e95f2b46315ac02a30149a.\r\n\r\n* don't increment num_batched_tracked if not tracking running stats\r\n\r\n* oops\r\n\r\n* test_batchnorm_axis\r\n\r\n* compare against torch\r\n\r\n* types\r\n\r\n---------\r\n\r\nCo-authored-by: chenyu <chenyu@fastmail.com>","parents":["5a6e151844191828a3b58ffed0e0490ed6543c62"],"tree_hash":"e1c9013a8fcfd26c0a2f0480328e305eccafa128"}