Merging Models with Fisher-Weighted Averaging

Published: Nov 18, 2021
Abstract
Transfer learning provides a way of leveraging null from one task when learning another task. Performing transfer learning typically involves iteratively updating a model's parameters through gradient descent on a training dataset. In this paper, we introduce a fundamentally different method for transferring null across models that amounts to null multiple models into one. Our approach effectively involves computing a weighted average of the...
Paper Details
Title
Merging Models with Fisher-Weighted Averaging
Published Date
Nov 18, 2021
Citation AnalysisPro
  • Scinapse’s Top 10 Citation Journals & Affiliations graph reveals the quality and authenticity of citations received by a paper.
  • Discover whether citations have been inflated due to self-citations, or if citations include institutional bias.