Skip to content

Commit 9d08a15

Browse files
committed
[revision] update README exp
1 parent 6edcfb3 commit 9d08a15

File tree

4 files changed

+534
-193506
lines changed

4 files changed

+534
-193506
lines changed

README.md

Lines changed: 19 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -285,7 +285,7 @@ We choose $\alpha = [0.1, 0.3, 0.5, 0.7]$ in label Dirichlet partitioned mnist w
285285
<p align="center"><img src="./examples/imgs/non_iid_impacts_on_fedavg.jpg" height="300"></p>
286286

287287

288-
The rounds for FedAvg to achieve 97% test accuracy on MNIST using 2NN with E=5 reported in [[4]](#4) / FedLab:
288+
We use the same partitioned MNIST dataset in FedAvg[[4]](#4) to evaluate the corectness of FedLab. The rounds for FedAvg to achieve 97% test accuracy on MNIST using 2NN with E=5 reported in [[4]](#4) / FedLab:
289289
<table>
290290
<tr>
291291
<td rowspan="2">Sample ratio</td>
@@ -302,56 +302,59 @@ The rounds for FedAvg to achieve 97% test accuracy on MNIST using 2NN with E=5 r
302302
<td>0.0</td>
303303
<td>1455 / 1293</td>
304304
<td>316 / 77 </td>
305-
<td>4278 / *</td>
305+
<td>4278 / 1815</td>
306306
<td>3275 / 1056</td>
307307
</tr>
308308
<tr>
309309
<td>0.1 </td>
310310
<td>1474 / 1230</td>
311311
<td>87 / 43 </td>
312-
<td>1796 / *</td>
312+
<td>1796 / 2778</td>
313313
<td>664 / 439</td>
314314
</tr>
315315
<tr>
316316
<td>0.2</td>
317317
<td>1658 / 1234</td>
318318
<td>77 / 37 </td>
319-
<td>1528 / *</td>
319+
<td>1528 / 2805</td>
320320
<td>619 / 427 </td>
321321
</tr>
322322
<tr>
323323
<td>0.5</td>
324-
<td>-- / *</td>
324+
<td>-- / 1229</td>
325325
<td>75 / 36 </td>
326-
<td>-- / *</td>
326+
<td>-- / 3034</td>
327327
<td>443 / 474</td>
328328
</tr>
329329
<tr>
330330
<td>1.0</td>
331-
<td>-- / *</td>
331+
<td>-- / 1284</td>
332332
<td>70 / 35 </td>
333-
<td>-- / *</td>
333+
<td>-- / 3154</td>
334334
<td>380 / 507</td>
335335
</tr>
336336
</table>
337337

338+
### Computation Efficienty
338339

339-
Time cost in 100 rounds under diffrent acceleration set(TODO):
340+
Time cost in 100 rounds (50 clients are sampled per round) under different acceleration settings. 1M-10P stands for the simulation runs on 1 machine with 4 GPUs and 10 processes. 2M-10P stands for the simulation runs on 2 machine with 4 GPUs and 10 processes (5 processes on each machine).
340341

341-
| Standalone | Cross-process 1M-4GPU-10P | Cross-process 2M-4*2GPU-10P |
342+
Hardware: Intel(R) Xeon(R) Gold 6240L CPU @ 2.60GHz + Tesla V100 * 4.
343+
344+
| Standalone | Cross-process 1M-10P | Cross-process 2M-10P |
342345
| ---------- | ------------------------- | --------------------------- |
343-
| * | * | * |
346+
| 45.6 Min | 2.9 Min | 4.23 Min |
344347

345348

346349

347350
### Communication Efficiency
348351

349-
We provide a few performance baseline in communication-efficient federated learning, which includes QSGD and top-k.
352+
We provide a few performance baselines in communication-efficient federated learning including QSGD and top-k. In the experiment setting, we choose $\alpha = 0.5$ in label Dirichlet partitioned mnist with 100 clients. We run 200 rounds with sample ratio 0.1 (10 clients for each FL round) of FedAvg, where each client performs 5 local epoches of SGD with full batch and learning rate 0.1. We report the top-1 test accuracy and its communication volume during the training.
350353

351-
| Setting | Baseline | QSGD-4bit | QSGD-8bit | QSGD-16bit | top-5% | Top-10% |
352-
| -------------------- | -------- | --------- | --------- | ---------- | ------ | ------- |
353-
| Test Accuracy | | | | | | |
354-
| Communication | | | | | | |
354+
| Setting | Baseline | QSGD-4bit | QSGD-8bit | QSGD-16bit | top-5% | Top-10% | Top-20% |
355+
| -------------------- | -------- | --------- | --------- | ---------- | ------ | ------- | ------- |
356+
| Test Accuracy (%) | 93.14 | 93.03 | 93.27 | 93.11 | 11.35 | 61.25 | 89.96 |
357+
| Communication (MB) | 302.45 | 45.59 | 85.06 | 160.67 | 0.94 | 1.89 | 3.79 |
355358

356359
## Citation
357360

0 commit comments

Comments
 (0)