Add Flower Baseline: Ditto

[oscardilley]

Paper

"Ditto: Fair and Robust Federated Learning Through Personalization" by Tian Li, Shengyuan Hu, Ahmad Beirami, Virginia Smith

Link

https://arxiv.org/abs/2012.04221

Maybe give motivations about why the paper should be implemented as a baseline.

From what I can see it was proposed for the summer of reproducibility but has not yet been contributed:

#2026 (comment)

I have implemented this as a part of other research and would like to contribute the work.

Is there something else you want to add?

No response

Implementation

To implement this baseline, it is recommended to do the following items in that order:

For first time contributors

• Read the first contribution doc
• Complete the Flower tutorial
• Read the Flower Baselines docs to get an overview:
  • How to use Flower Baselines
  • How to contribute a Flower Baseline

Prepare - understand the scope

• Read the paper linked above
• Decide which experiments you'd like to reproduce. The more the better!
• Follow the steps outlined in Add a new Flower Baseline.
• You can use as reference other baselines that the community merged following those steps.

Verify your implementation

• Follow the steps indicated in the EXTENDED_README.md that was created in your baseline directory
• Ensure your code reproduces the results for the experiments you chose
• Ensure your README.md is ready to be run by someone that is no familiar with your code. Are all step-by-step instructions clear?
• Ensure running the formatting and typing tests for your baseline runs without errors.
• Clone your repo on a new directory, follow the guide on your own README.md and verify everything runs.

[jafermarq]

Hi @oscardilley your are right, Ditto was one of the baselines proposed for Summer of Reproducibility but it wasn't completed. Would you like to add it to Flower Baselines? Do you need any help?

[oscardilley]

Hey @jafermarq, yes, I would really like to contribute it as a baseline - I have started reconfiguring it into the required format. I think I am getting on well so far, but is this the best channel to ask any future questions I may have?

[oscardilley]

Hi @jafermarq , The reproduction of Ditto is going well, but I have a couple of sticking points that I was hoping you could advise on... From the paper <https://arxiv.org/abs/2012.04221> + repo <https://github.com/litian96/ditto>, they don't seem to provide some of hyperparameters used to achieve the reported results, specifically: - Whether the stated learning rate applies to the local and global objectives, in relation to Algorithm 2. - The number of clients selected for each round of training. - The number of local iterations to update the global and personalised parameters. With this in mind, what level of result reproduction do you expect? Further, would it be acceptable, for the contribution, to only replicate a subset of the results. i.e. the 'clean' tests without clients acting maliciously? Thanks!

…

On Fri, Jul 12, 2024 at 10:05 AM Javier ***@***.***> wrote: Hi @oscardilley <https://github.com/oscardilley> your are right, Ditto was one of the baselines proposed for Summer of Reproducibility but it wasn't completed. Would you like to add it to Flower Baselines? Do you need any help? — Reply to this email directly, view it on GitHub <#3756 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AO72UZXJDXILZSTV4B6JYSLZL6L6PAVCNFSM6AAAAABKS7L6X6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRVGE2DONBSGY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[oscardilley]

Hi, @jafermarq, please could you advise on this?

[mercurius80]

Hi @oscardilley, I am new here but work already in ML Field with flower. I would like to contribute. Do you need any help? Where can I start?
Cheers

[WilliamLindskog]

Hi @oscardilley, @mercurius80,

Just checking in here. Are there any updates or is there anything we can do to help?

[WilliamLindskog]

It seems that in their repo, they treat the learning rate as one (local and global are the same), I can't find code that distinguishes between them both. 20 clients seem to be default but that they sample 10 of them (fraction 50%). Local iterations seem to be set to 2 and something called finetune-iters is 40. You might have already seen this?

[oscardilley]

It seems that in their repo, they treat the learning rate as one (local and global are the same), I can't find code that distinguishes between them both. 20 clients seem to be default but that they sample 10 of them (fraction 50%). Local iterations seem to be set to 2 and something called finetune-iters is 40. You might have already seen this?

Hi @WilliamLindskog, thanks for reaching out. Sorry for the delay on this, I have been busy with work. Good spot on those parameters. Following up from a previous question that I had - for it to be an acceptable reproduction/ baseline, how many/ which of the plots from the paper would it be ideal to reproduce?

I am hoping to have some time to pick this back up in April.

[WilliamLindskog]

@jafermarq any thoughts on how many plots? I'd say the more the merrier, however Figure 4 and 6 should be included I'd say.

[jafermarq]

@oscardilley @WilliamLindskog with Baselines we typically aim to have the core results reproduced. In the case of Ditto I see those in Table 1. Having a plot out of these (or even additional experiments is nice) but probably is better to focus on the table. It should be fine having adversaries ratios 20% and 80%. How does that sound ?

[oscardilley]

Sounds great, thanks. I will crack on in April time and submit a PR. On 29 Jan 2025, at 17:45, Javier ***@***.***> wrote: @oscardilley @WilliamLindskog with Baselines we typically aim to have the core results reproduced. In the case of Ditto I see those in Table 1. Having a plot out of these (or even additional experiments is nice) but probably is better to focus on the table. It should be fine having adversaries ratios 20% and 80%. How does that sound ? —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>