Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

speedup 'hf mf chk' #901

Merged
merged 6 commits into from
Jan 9, 2020
Merged

speedup 'hf mf chk' #901

merged 6 commits into from
Jan 9, 2020

Conversation

pwpiwi
Copy link
Contributor

@pwpiwi pwpiwi commented Dec 23, 2019

  • add separate timeout for tag response to nr_ar
  • measure response time and use it for response timeout
  • don't drop field between keyblocks
  • some reformatting
  • some whitespace fixes

* add separate timeout for tag response to nr_ar
* measure response time and use it for response timeout
* don't drop field between keyblocks
* some reformatting
* some whitespace fixes
* fishing for microseconds in TransmitFor14443a()
* some reformatting
@@ -1284,9 +1273,6 @@ static void TransmitFor14443a(const uint8_t *cmd, uint16_t len, uint32_t *timing
LastTimeProxToAirStart = ThisTransferTime;
}

// clear TXRDY
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why remove this one?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sends a 0x00 byte just to clear TXREADY. In the following loop we are waiting for TXREADY to be set again. This wastes 9.4us.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@iceman1001
Copy link
Member

Tested the idea of setting / restoring timeout. Got 206/sec. Impressive speedups.

Comment on lines 1027 to 1041
if (res >= 0) {
cmd_send(CMD_ACK, 1, res, 0, keyIndex, 80);
} else {
cmd_send(CMD_ACK, 0, res, 0, NULL, 0);
drop_field = true;
}
} else {
int res = MifareChkBlockKeys(datain, keyCount, blockNo, keyType, OLD_MF_DBGLEVEL);
int res = MifareChkBlockKeys(datain, keyCount, blockNo, keyType, &auth_timeout, OLD_MF_DBGLEVEL);

if (res > 0) {
cmd_send(CMD_ACK, 1, res, 0, datain + (res - 1) * 6, 6);
} else {
cmd_send(CMD_ACK, 0, res, 0, NULL, 0);
drop_field = true;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this drop_field logic backwards? As written it always drops the field when the key is not found (and we're about to come back here again with a new set of keys to check) but does not drop it when the key is found and we're finished.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Absolutely. Pushed a fix.

@uzlonewolf
Copy link
Contributor

Field should be dropped on every return value except 0 as >0 is success and <0 is error

    } else {
	int res = MifareChkBlockKeys(datain, keyCount, blockNo, keyType, &auth_timeout, OLD_MF_DBGLEVEL);

        if (res != 0)
            drop_field = true;

        if (res > 0)
	    cmd_send(CMD_ACK, 1, res, 0, datain + (res - 1) * 6, 6);
        else
            cmd_send(CMD_ACK, 0, res, 0, NULL, 0);
    }

Before:
Key found in 361.69 seconds after checking 78118 keys

After:
Key found in 361.57 seconds after checking 78118 keys

We get even more by pulling the (incorrect) && !usb_poll_validate_length() out of int MifareChkBlockKeys():
Key found in 361.35 seconds after checking 78118 keys

And slightly more still by pulling the button press detection out of the loop completely.

// multi key check
int MifareChkBlockKeys(uint8_t *keys, uint8_t keyCount, uint8_t blockNo, uint8_t keyType, uint32_t *auth_timeout, uint8_t debugLevel) {

    uint8_t uid[10];
    uint32_t cuid = 0;
    uint8_t cascade_levels = 0;
    uint64_t ui64Key = 0;

    // Allow button press to interrupt device
    if (BUTTON_PRESS()) {
        Dbprintf("ChkKeys: Cancel operation. Exit...");
        return -2;
    }

    int retryCount = 0;
    for (uint8_t i = 0; i < keyCount; i++) {

        // Allow button press / usb cmd to interrupt device
        //if (BUTTON_PRESS() && !usb_poll_validate_length()) {
        //if (BUTTON_PRESS()) {
        //  Dbprintf("ChkKeys: Cancel operation. Exit...");
        //  return -2;
        //}

        ui64Key = bytes_to_num(keys + i * 6, 6);

@uzlonewolf
Copy link
Contributor

I was looking at the lights blink and thought a lot of time is wasted in waiting for the first set of keys to complete before transferring the next set, so I re-wrote it to "queue up" and USB transfer the next batch of keys while it's checking and got another ~16 keys/sec out of it.

...
     1618 keys left | 234.1 keys/sec | worst case    6.9 seconds remaining          
Key found in 334.17 seconds after checking 78118 keys

I'll put together a PR for this one.

@pwpiwi
Copy link
Contributor Author

pwpiwi commented Dec 29, 2019

Not sure if this is worth the effort. And I am exploring another check functions tailored to fixed nonces. This should be faster anyway (I hope).

@uzlonewolf
Copy link
Contributor

I like the new drop_field check however MifareMultisectorChk() needs to be modified to return 1 on success for it to work. I went ahead and made a PR @ pwpiwi#1 anyway since I was basically done. The keys/sec calculation needs some tweaking as it now thinks 1 block more than actual has been completed.

@pwpiwi
Copy link
Contributor Author

pwpiwi commented Dec 29, 2019

Hmm, I am getting lot's of connection errors:

#db# Nonce#2: cannot get nonce that != nonce#1, continuing anyway with single nonce! ntdist=160
uid:c862543c trgbl=4 trgkey=0
Setting authentication timeout to 103us
statelist 0: length:128110 block:04 keytype:0 nt:7EEF3586 ks1:9422BAF4
statelist 1: length:128110 block:04 keytype:0 nt:7EEF3586 ks1:9422BAF4
Nonce 1 and 2 are the same!
We have 128110 keys to check. This will take a very long time!
Press button to abort.
Sending bytes to proxmark failed
Sending bytes to proxmark failed
Sending bytes to proxmark failed
   127770 keys left | 279.6 keys/sec | worst case  457.0 seconds remaining
Sending bytes to proxmark failed
Sending bytes to proxmark failed
Sending bytes to proxmark failed
Sending bytes to proxmark failed
Sending bytes to proxmark failed
Sending bytes to proxmark failed
Sending bytes to proxmark failed
Sending bytes to proxmark failed
Sending bytes to proxmark failed
Sending bytes to proxmark failed
Sending bytes to proxmark failed
Sending bytes to proxmark failed
Sending bytes to proxmark failed
Sending bytes to proxmark failed

@pwpiwi
Copy link
Contributor Author

pwpiwi commented Dec 29, 2019

Probably because there is no way for the Proxmark to tell the client when it is ready to receive another bunch of keys. And it cannot receive any commands while it is working on the current key block,..

@uzlonewolf
Copy link
Contributor

uzlonewolf commented Dec 29, 2019

Really? It's working fine on mine.

proxmark3> hf mf nested o 0 A a0a1a2a3a4a5 4 A
--nested. sectors: 1, block no:  0, key type:A, eml:n, dmp=n checktimeout=471 us          
--target block no:  4, target key type:A           
#db# Nonce#2: cannot get nonce that != nonce#1, continuing anyway with single nonce! ntdist=160          
uid:2a0axxxx trgbl=4 trgkey=0          
Setting authentication timeout to 103us          
statelist 0: length:67272 block:04 keytype:0 nt:7EEF3586 ks1:802DEAB4          
statelist 1: length:67272 block:04 keytype:0 nt:7EEF3586 ks1:802DEAB4          
Nonce 1 and 2 are the same!          
We have 67272 keys to check. This will take a very long time!          
Press button to abort.          
    66932 keys left | 284.3 keys/sec | worst case  235.4 seconds remaining          
Key found in 9.04 seconds after checking 2125 keys
          
Found valid key:2a010a10xxxx          
proxmark3> hf mf nested o 0 A a0a1a2a3a4a5 24 A
--nested. sectors: 1, block no:  0, key type:A, eml:n, dmp=n checktimeout=471 us          
--target block no: 24, target key type:A           
#db# Nonce#2: cannot get nonce that != nonce#1, continuing anyway with single nonce! ntdist=160          
uid:2a0axxxx trgbl=24 trgkey=0          
Setting authentication timeout to 103us          
statelist 0: length:78118 block:24 keytype:0 nt:7EEF3586 ks1:FF37BECC          
statelist 1: length:78118 block:24 keytype:0 nt:7EEF3586 ks1:FF37BECC          
Nonce 1 and 2 are the same!          
We have 78118 keys to check. This will take a very long time!          
Press button to abort.          
    77778 keys left | 287.2 keys/sec | worst case  270.9 seconds remaining          
    75398 keys left | 239.8 keys/sec | worst case  314.5 seconds remaining          
    73018 keys left | 237.2 keys/sec | worst case  307.9 seconds remaining          
    70638 keys left | 235.9 keys/sec | worst case  299.4 seconds remaining          
    68258 keys left | 235.6 keys/sec | worst case  289.7 seconds remaining          
    65878 keys left | 235.1 keys/sec | worst case  280.2 seconds remaining          
    63498 keys left | 234.8 keys/sec | worst case  270.4 seconds remaining          
    61118 keys left | 234.7 keys/sec | worst case  260.4 seconds remaining          
    58738 keys left | 234.7 keys/sec | worst case  250.3 seconds remaining          
    56358 keys left | 234.7 keys/sec | worst case  240.1 seconds remaining          
    53978 keys left | 234.7 keys/sec | worst case  230.0 seconds remaining          
    51598 keys left | 234.6 keys/sec | worst case  219.9 seconds remaining          
    49218 keys left | 234.6 keys/sec | worst case  209.8 seconds remaining          
    46838 keys left | 234.5 keys/sec | worst case  199.7 seconds remaining          
    44458 keys left | 234.4 keys/sec | worst case  189.6 seconds remaining          
    42078 keys left | 234.4 keys/sec | worst case  179.5 seconds remaining          
    39698 keys left | 234.3 keys/sec | worst case  169.4 seconds remaining          
    37318 keys left | 234.3 keys/sec | worst case  159.3 seconds remaining          
    34938 keys left | 234.3 keys/sec | worst case  149.1 seconds remaining          
    32558 keys left | 234.3 keys/sec | worst case  139.0 seconds remaining          
    30178 keys left | 234.3 keys/sec | worst case  128.8 seconds remaining          
    27798 keys left | 234.3 keys/sec | worst case  118.7 seconds remaining          
    25418 keys left | 234.2 keys/sec | worst case  108.5 seconds remaining          
    23038 keys left | 234.2 keys/sec | worst case   98.4 seconds remaining          
    20658 keys left | 234.2 keys/sec | worst case   88.2 seconds remaining          
    18278 keys left | 234.2 keys/sec | worst case   78.0 seconds remaining          
    15898 keys left | 234.2 keys/sec | worst case   67.9 seconds remaining          
    13518 keys left | 234.2 keys/sec | worst case   57.7 seconds remaining          
    11138 keys left | 234.2 keys/sec | worst case   47.6 seconds remaining          
     8758 keys left | 234.2 keys/sec | worst case   37.4 seconds remaining          
     6378 keys left | 234.1 keys/sec | worst case   27.2 seconds remaining          
     3998 keys left | 234.1 keys/sec | worst case   17.1 seconds remaining          
     1618 keys left | 234.1 keys/sec | worst case    6.9 seconds remaining          
Key found in 334.05 seconds after checking 78118 keys
          
Found valid key:ffffffffffff          

I thought the USB side was a hardware peripheral? Sure it can't process it while working, but I thought it would just sit in the buffer in that case.

@pwpiwi
Copy link
Contributor Author

pwpiwi commented Dec 29, 2019

I thought the USB side was a hardware peripheral? Sure it can't process it while working, but I thought it would just sit in the buffer in that case.

The hardware buffer is only 64 Bytes. Maybe the Operating System makes the difference here. You are on Linux? I am on Windows.

@uzlonewolf
Copy link
Contributor

Yep, on an antique version of Fedora.

@pwpiwi
Copy link
Contributor Author

pwpiwi commented Dec 29, 2019

Confirmed. Works on Kali without any changes.

The speed measurement for first keyblock is still off (but now it is too fast instead of too slow).

@pwpiwi
Copy link
Contributor Author

pwpiwi commented Dec 29, 2019

Different timeouts or error handling in uart_win32.c vs uart_posix.c ?

@uzlonewolf
Copy link
Contributor

Possibly, I remember those files were way different and when I was working on disconnect/reconnect they needed some changes. I don't have a Windows box so I can't test easily.

@pwpiwi
Copy link
Contributor Author

pwpiwi commented Dec 29, 2019

Which reminds me of PR#516 which is still open. Can you rebase it?

@uzlonewolf
Copy link
Contributor

Well, it turns out that moving the button press check doesn't work so well. I put it back and fixed the keys/sec but when trying to make a new PR it's wanting to pull all the changes in again. pwpiwi@3b62d28

I'll see if I can get to #516 in the next day or 2.

* revert queuing change
* instead allow arbitrary number of keys in MifareChkKeys()
* and move progress printing to MifareChkKeys()
@pwpiwi
Copy link
Contributor Author

pwpiwi commented Dec 30, 2019

For the time being I have reverted your queuing patch. I propose to merge this PR (if there aren't any more issues) and open a new one for queuing when the comms issue is sorted out.

* fix abort by pressing the PM3 button
@pwpiwi pwpiwi merged commit a749b1e into Proxmark:master Jan 9, 2020
@uzlonewolf
Copy link
Contributor

Yep, sounds good. I'll try to get to both that and #516, though I'm not sure when as I'm back home and don't have much free time at the moment.

@pwpiwi pwpiwi deleted the speedup_mf_chk branch January 9, 2020 17:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants