Summary

• States LTH's method of finding a winning ticket is not necessary
• It's commonly believed in literature that you have to: train, then prune and then fine-tune
• But this is not necessary for unstructured pruning
• Instead, you can find a better set of hparams for the optimization method, e.g. for SGD
• For unstructured pruning this does not scale to ImageNet. They do not know why. Only comparable perf for smaller datasets (and models?)