FedNereus: Incentive-Aware Heterogeneous Federated Learning Under Resource Constraint

Abstract

As public concern about data privacy continues to grow, federated learning (FL), as a machine learning technique that does not require the disclosure of user raw data, has attracted widespread attention. However, it still has many common problems. The first problem stems from the absence of incentive mechanisms utilized to incentivize more participation to improve the performance of learning models. The second problem arises from the limited resources of mobile devices that can be utilized for model training, which will seriously decrease the performance of learning models. To address the aforementioned issues, in this paper, we propose a novel incentive-aware collaborative construction of heterogeneous <underline>Fed</underline>erated lear<underline>N</underline>ing und<underline>e</underline>r <underline>re</underline>so<underline>u</underline>rce con<underline>s</underline>traint, namely FedNereus. In fact, FedNereus incentivizes more workers to participate in model training by applying an auction-based incentive mechanism. It is proved that FedNereus allows participating workers to report their actual cost as their bidding price, which is referred to as <italic>truthfulness</italic>. Furthermore, it also allows workers to obtain non-negative reward, which is also referred to as the <italic>individual rationality</italic>. In order to break the resource limitation, FedNereus meticulously designs the rules of model selection so that workers can only train a portion of the learning model based on their limited hardware capabilities, while maintaining the performance guarantee of learning models. The excess empirical risk of FedNereus is shown to be upper bounded by <inline-formula><tex-math notation="LaTeX">$\mathcal{O}(\frac{1}{T})$</tex-math><alternatives><mml:math><mml:mrow><mml:mi mathvariant="script">O</mml:mi></mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mfrac><mml:mn>1</mml:mn><mml:mi>T</mml:mi></mml:mfrac><mml:mo stretchy="false">)</mml:mo></mml:math><inline-graphic xlink:href="ying-ieq1-3713425.gif"/></alternatives></inline-formula>, where <inline-formula><tex-math notation="LaTeX">$T$</tex-math><alternatives><mml:math><mml:mi>T</mml:mi></mml:math><inline-graphic xlink:href="ying-ieq2-3713425.gif"/></alternatives></inline-formula> is the number of training rounds. Finally, extensive experiments are conducted, whose results show that FedNereus outperforms state-of-the-art approaches in different learning tasks.