solve the problem of failed parsing of STL containers #433

HenryRiley0 · 2024-07-22T09:58:35Z

solve the problem of failed parsing of STL containers

GitMensch · 2024-08-22T13:19:06Z

As the next entry in that loop is the parsing of primitives and that cannot include text I suggest to not check for std:: but instead for indexOf(" = {"), this should also work with derived container classes. What do you think?

I do wonder what the output of a container containing containers look like. Can you please test?

Can you try to replace "" by the part before the = { like "std::vector of length 4, capacity 6" as well?

In any case please add the Changelog entry.

HenryRiley0 · 2024-08-23T03:30:21Z

I do wonder what the output of a container containing containers look like. Can you please test?
like below, and it works fine:

18-data-evaluate-expression "vecOfVecs"
GDB -> App: {"output":"","token":18,"outOfBandRecord":[],"resultRecords":{"resultClass":"done","results":[["value","std::vector of length 4, capacity 6 = {std::vector of length 3, capacity 3 = {1, 2, 3}, std::vector of length 3, capacity 3 = {4, 5, 6}, std::vector of length 3, capacity 3 = {7, 8, 9}, std::vector of length 4, capacity 4 = {1, 2, 3, 4}}"]]}}
19-data-evaluate-expression "deqOfDeqs"
GDB -> App: {"output":"","token":19,"outOfBandRecord":[],"resultRecords":{"resultClass":"done","results":[["value","std::deque with 3 elements = {std::deque with 2 elements = {10, 20}, std::deque with 2 elements = {30, 40}, std::deque with 7 elements = {0, 1, 2, 3, 4, 5, 6}}"]]}}
20-data-evaluate-expression "listOfLists"
GDB -> App: {"output":"","token":20,"outOfBandRecord":[],"resultRecords":{"resultClass":"done","results":[["value","std::__cxx11::list = {[0] = std::__cxx11::list = {[0] = 11, [1] = 12}, [1] = std::__cxx11::list = {[0] = 13, [1] = 14}, [2] = std::__cxx11::list = {[0] = 6, [1] = 7, [2] = 8, [3] = 9, [4] = 10}}"]]}}

HenryRiley0 · 2024-08-23T06:39:17Z

As the next entry in that loop is the parsing of primitives and that cannot include text I suggest to not check for std:: but instead for indexOf(" = {"), this should also work with derived container classes. What do you think?

it seems check for indexOf(" = {") may hit an object, in the middle or back of an value string

GitMensch · 2024-08-23T06:54:51Z

You're right.

if (value.match(/^[^=]* = /) {
    const eqPos = valuie.indexOf("="):
    value = value.substring(eqPos + 2)):
    return parseValue():
}

should fix that and drop the trim.

GitMensch · 2024-08-23T07:04:17Z

Please rebase, so that we have only one (squashed) or two commits (which I'd use the "merge squashed" operation for) and a clearer diff.

HenryRiley0 · 2024-08-23T07:57:21Z

You're right.

if (value.match(/^[^=]* = /) {
    const eqPos = valuie.indexOf("="):
    value = value.substring(eqPos + 2)):
    return parseValue():
}

should fix that and drop the trim.

18-data-evaluate-expression "vec"
GDB -> App: {"token":18,"outOfBandRecord":[],"resultRecords":{"resultClass":"done","results":[["value","std::vector of length 4, capacity 6 = {1, 2, 3, 4}"]]}}
19-data-evaluate-expression "deq"
GDB -> App: {"token":19,"outOfBandRecord":[],"resultRecords":{"resultClass":"done","results":[["value","std::deque with 7 elements = {0, 1, 2, 3, 4, 5, 6}"]]}}
20-data-evaluate-expression "myList"
GDB -> App: {"token":20,"outOfBandRecord":[],"resultRecords":{"resultClass":"done","results":[["value","std::__cxx11::list = {[0] = 6, [1] = 7, [2] = 8, [3] = 9, [4] = 10}"]]}}
21-data-evaluate-expression "vecOfVecs"
GDB -> App: {"token":21,"outOfBandRecord":[],"resultRecords":{"resultClass":"done","results":[["value","std::vector of length 4, capacity 6 = {std::vector of length 3, capacity 3 = {1, 2, 3}, std::vector of length 3, capacity 3 = {4, 5, 6}, std::vector of length 3, capacity 3 = {7, 8, 9}, std::vector of length 4, capacity 4 = {1, 2, 3, 4}}"]]}}
22-data-evaluate-expression "deqOfDeqs"
GDB -> App: {"token":22,"outOfBandRecord":[],"resultRecords":{"resultClass":"done","results":[["value","std::deque with 3 elements = {std::deque with 2 elements = {10, 20}, std::deque with 2 elements = {30, 40}, std::deque with 7 elements = {0, 1, 2, 3, 4, 5, 6}}"]]}}

i tested only vec and deq works, myList、vecOfVecs、deqOfDeqs parse fail.

GitMensch · 2024-08-23T09:12:51Z

i tested only vec and deq works, myList、vecOfVecs、deqOfDeqs parse fail.

What is the mi2 output which fails to parse?

Does it work (at least as good as the current version) when replacing the std:: check with (value.match(/^[^=]* = /) {

HenryRiley0 · 2024-08-23T10:23:36Z

it's strange

sucess

		else if(value.startsWith("std::")){
			const eqPos = value.indexOf("=");
			value = value.substring(eqPos + 2);
			return parseValue();
		}

fail

		else if(value.match(/^[^=]* = /)){
			const eqPos = value.indexOf("=");
			value = value.substring(eqPos + 2);
			return parseValue();
		}

test code:, break point 139

#include <iostream>
#include <vector>
#include <deque>
#include <list>
#include <forward_list>
#include <set>
#include <map>
#include <unordered_set>
#include <unordered_map>
#include <stack>
#include <queue>
#include <bitset>
#include <array>
#include <memory>
#include <atomic>
#include <fstream>
#include <sstream>
#include <regex>
#include <algorithm>
#include <mutex>

void testSTLContainers() {
    // std::vector
    std::vector<int> vec = {1, 2, 3};
    vec.push_back(4);

    // std::deque
    std::deque<int> deq = {1, 2, 3, 4, 5};
    deq.push_front(0);
    deq.push_back(6);

    // std::list
    std::list<int> myList = {7, 8, 9};
    myList.push_back(10);
    myList.push_front(6);

    std::vector<std::vector<int>> vecOfVecs = {{1, 2, 3}, {4, 5, 6}, {7, 8, 9}};
    vecOfVecs.push_back(vec);

    std::deque<std::deque<int>> deqOfDeqs = {{10, 20}, {30, 40}};
    deqOfDeqs.push_back(deq);

    std::list<std::list<int>> listOfLists = {{11, 12}, {13, 14}};
    listOfLists.push_back(myList);

    // std::forward_list
    std::forward_list<int> fwdList = {1, 2, 3};
    fwdList.push_front(0);

    // std::set
    std::set<int> mySet = {4, 1, 3, 2};
    mySet.insert(5);

    // std::multiset
    std::multiset<int> myMultiSet = {1, 2, 2, 3, 4};
    myMultiSet.insert(2);

    // std::map
    std::map<int, std::string> myMap = {{1, "one"}, {2, "two"}};
    myMap[3] = "three";

    // std::multimap
    std::multimap<int, std::string> myMultiMap = {{1, "one"}, {2, "two"}, {2, "another two"}};

    // std::unordered_set
    std::unordered_set<int> myUnorderedSet = {4, 1, 3, 2};
    myUnorderedSet.insert(5);

    // std::unordered_multiset
    std::unordered_multiset<int> myUnorderedMultiSet = {1, 2, 2, 3, 4};
    myUnorderedMultiSet.insert(2);

    // std::unordered_map
    std::unordered_map<int, std::string> myUnorderedMap = {{1, "one"}, {2, "two"}};
    myUnorderedMap[3] = "three";

    // std::unordered_multimap
    std::unordered_multimap<int, std::string> myUnorderedMultiMap = {{1, "one"}, {2, "two"}, {2, "another two"}};

    // std::stack (based on std::deque)
    std::stack<int> myStack;
    myStack.push(1);
    myStack.push(2);

    // std::queue (based on std::deque)
    std::queue<int> myQueue;
    myQueue.push(1);
    myQueue.push(2);

    // std::priority_queue (max heap)
    std::priority_queue<int> myMaxHeap;
    myMaxHeap.push(3);
    myMaxHeap.push(1);
    myMaxHeap.push(4);

    // std::bitset
    std::bitset<8> bits("11010101");

    // std::array
    std::array<int, 3> myArray = {1, 2, 3};

    // std::unique_ptr
    std::unique_ptr<int> uniquePtr(new int(42));

    std::unique_ptr<int[]> uniquePtr1(new int[2]);

    uniquePtr1[0] = 10; 
    uniquePtr1[1] = 20; 

    // std::shared_ptr
    std::shared_ptr<int> sharedPtr = std::make_shared<int>(10);

    // std::weak_ptr
    std::weak_ptr<int> weakPtr = sharedPtr;

    // std::atomic
    std::atomic<int> atomicInt(0);

    // std::mutex
    std::mutex mtx;

    // std::ofstream
    std::ofstream file("example.txt");
    file << "Hello, world!";
    file.close();

    // std::stringstream
    std::stringstream ss;
    ss << "The answer is: " << 42;
    std::string result = ss.str();

    // std::regex
    std::regex pattern("\\d+");
    std::string text = "Hello 123 world";
    std::smatch match;
    std::regex_search(text, match, pattern);

    // Output results
    std::cout << "Vector elements: ";
    for (const auto& v : vec) {
        std::cout << v << " ";
    }
    std::cout << std::endl;

    std::cout << "Deque elements: ";
    for (const auto& d : deq) {
        std::cout << d << " ";
    }
    std::cout << std::endl;

    std::cout << "List elements: ";
    for (const auto& l : myList) {
        std::cout << l << " ";
    }
    std::cout << std::endl;

    std::cout << "Forward List elements: ";
    for (const auto& fl : fwdList) {
        std::cout << fl << " ";
    }
    std::cout << std::endl;

    std::cout << "Set elements: ";
    for (const auto& s : mySet) {
        std::cout << s << " ";
    }
    std::cout << std::endl;

    std::cout << "MultiSet elements: ";
    for (const auto& ms : myMultiSet) {
        std::cout << ms << " ";
    }
    std::cout << std::endl;

    std::cout << "Map elements: ";
    for (const auto& m : myMap) {
        std::cout << "{" << m.first << ": " << m.second << "} ";
    }
    std::cout << std::endl;

    std::cout << "MultiMap elements: ";
    for (const auto& mm : myMultiMap) {
        std::cout << "{" << mm.first << ": " << mm.second << "} ";
    }
    std::cout << std::endl;

    std::cout << "Unordered Set elements: ";
    for (const auto& us : myUnorderedSet) {
        std::cout << us << " ";
    }
    std::cout << std::endl;

    std::cout << "Unordered MultiSet elements: ";
    for (const auto& ums : myUnorderedMultiSet) {
        std::cout << ums << " ";
    }
    std::cout << std::endl;

    std::cout << "Unordered Map elements: ";
    for (const auto& um : myUnorderedMap) {
        std::cout << "{" << um.first << ": " << um.second << "} ";
    }
    std::cout << std::endl;

    std::cout << "Unordered MultiMap elements: ";
    for (const auto& umm : myUnorderedMultiMap) {
        std::cout << "{" << umm.first << ": " << umm.second << "} ";
    }
    std::cout << std::endl;

    std::cout << "Stack elements: ";
    while (!myStack.empty()) {
        std::cout << myStack.top() << " ";
        myStack.pop();
    }
    std::cout << std::endl;

    std::cout << "Queue elements: ";
    while (!myQueue.empty()) {
        std::cout << myQueue.front() << " ";
        myQueue.pop();
    }
    std::cout << std::endl;

    std::cout << "Priority Queue elements (max heap): ";
    while (!myMaxHeap.empty()) {
        std::cout << myMaxHeap.top() << " ";
        myMaxHeap.pop();
    }
    std::cout << std::endl;

    std::cout << "Bitset value: " << bits << std::endl;

    std::cout << "Array elements: ";
    for (const auto& a : myArray) {
        std::cout << a << " ";
    }
    std::cout << std::endl;

    std::cout << "Unique pointer value: " << *uniquePtr << std::endl;
    std::cout << "Shared pointer value: " << *sharedPtr << std::endl;
    std::cout << "Atomic integer value: " << atomicInt << std::endl;
    std::cout << "Regex match: " << match[0] << std::endl;
}

int main() {
    testSTLContainers();
    return 0;
}

HenryRiley0 · 2024-08-23T10:28:19Z

pls dont merge, there is still a bug. i will fix next week.

HenryRiley0 · 2024-08-26T06:24:01Z

work fine:

GDB -> App: {"token":18,"outOfBandRecord":[],"resultRecords":{"resultClass":"done","results":[["value","std::deque with 7 elements = {0, 1, 2, 3, 4, 5, 6}"]]}}
19-data-evaluate-expression "vec"
GDB -> App: {"token":19,"outOfBandRecord":[],"resultRecords":{"resultClass":"done","results":[["value","std::vector of length 4, capacity 6 = {1, 2, 3, 4}"]]}}
20-data-evaluate-expression "myList"
GDB -> App: {"token":20,"outOfBandRecord":[],"resultRecords":{"resultClass":"done","results":[["value","std::__cxx11::list = {[0] = 6, [1] = 7, [2] = 8, [3] = 9, [4] = 10}"]]}}
21-data-evaluate-expression "vecOfVecs"
GDB -> App: {"token":21,"outOfBandRecord":[],"resultRecords":{"resultClass":"done","results":[["value","std::vector of length 4, capacity 6 = {std::vector of length 3, capacity 3 = {1, 2, 3}, std::vector of length 3, capacity 3 = {4, 5, 6}, std::vector of length 3, capacity 3 = {7, 8, 9}, std::vector of length 4, capacity 4 = {1, 2, 3, 4}}"]]}}
22-data-evaluate-expression "deqOfDeqs"
GDB -> App: {"token":22,"outOfBandRecord":[],"resultRecords":{"resultClass":"done","results":[["value","std::deque with 3 elements = {std::deque with 2 elements = {10, 20}, std::deque with 2 elements = {30, 40}, std::deque with 7 elements = {0, 1, 2, 3, 4, 5, 6}}"]]}}

GitMensch · 2024-08-26T13:44:18Z

Do you have any idea what's wrong with if (value.match(/^[^=]* = /)) {?

HenryRiley0 · 2024-08-27T02:54:06Z

Do you have any idea what's wrong with if (value.match(/^[^=]* = /)) {?

std::__cxx11::list = {[0] = 6, [1] = 7, [2] = 8, [3] = 9, [4] = 10}

[0] = 6, [1] = 7, [2] = 8, [3] = 9, [4] = 10}

value.match(/^[^=]* = /) works fine for std, but "[0] = 6, [1] = 7, [2] = 8, [3] = 9, [4] = 10}" will be match

GitMensch · 2024-09-13T21:54:34Z

Sounds like if (value.match(/^[^ =]* = /)) (= stopping at the first space) would be a better match.
The reason for this pickiness here is that we'd like to also handle non-std containers correctly.

HenryRiley0 · 2024-09-14T03:12:37Z

Sounds like if (value.match(/^[^ =]* = /)) (= stopping at the first space) would be a better match. The reason for this pickiness here is that we'd like to also handle non-std containers correctly.

Yes, I have tried it. The "value.match(/^[^ =]* = /)" matches "std::* =", but it also matches other strings, such as _M_elems = {1, 2, 3} and [0] = 6, [1] = 7, [2] = 8, [3] = 9, [4] = 10, etc.
In the test example, [["value","{_M_elems = {1, 2, 3}}"]], the match of "_M_elems = {1, 2, 3}" results in [0] = {1, 2, 3};
The match of "[0] = 6, [1] = 7, [2] = 8, [3] = 9, [4] = 10" in std::__cxx11::list = {[0] = 6, [1] = 7, [2] = 8, [3] = 9, [4] = 10} results in [0] = 10.
There are also other errors that prevent value.startsWith("std::") from being directly replaced with value.match(/^[^ =]* = /), because value.match(/^[^ =]* = /) broadens the scope and can lead to some logical errors.

GitMensch · 2024-09-18T09:19:13Z

Can you please add a Changelog entry and squash the commits to one?

HenryRiley0 · 2024-09-18T09:45:27Z

Can you please add a Changelog entry and squash the commits to one?

Of course, I will handle it right away.

GitMensch

Thanks for your bugfix!

The unwanted revert needs a fix, the "tmp" part should be adjusted for keeping the style.
It is sad, that your useful examples will only stay in the issue - it would be nice if we could add that to the testsuite - but I haven't had a deeper look into that...

Maybe add the samples at least in the detail part of the commit message?

src/backend/gdb_expansion.ts

src/mibase.ts

GitMensch

Thanks for the testcase adjustment! All code looks fine to me but I'm confused about the lamda input and return. Can you please explain that to me?

GitMensch · 2024-09-19T09:12:11Z

src/backend/gdb_expansion.ts

+		value = value.trim();
+		if (value[0] == ','){
+			let tmp = value;
+			tmp =  tmp.substring(1).trim();
+			if(tmp.startsWith("<No data fields>")){
+				value = tmp.replace(/^<No data fields>/, "");
+			}
+		}


Isn't that just the following?

value = value.replace(/^, <No data fields>/, "");

The one question open - I haven't recognized that before: value is an outer variable, maybe it should be passed in via the lambda and also be returned?

But I'm also not sure: How is value:val in the return related to changing value directly?

value = value.replace(/^, /, "");
that's cool. i tested, work fine.

The value of val has already been correctly obtained before the processing of value = value.replace(/^, /, "");. The processing with value = value.replace(/^, /, ""); is necessary because not removing ", " will affect the subsequent parsing.

GitMensch · 2024-09-19T09:48:05Z

Nice work! On to the next issue :-)

GitMensch changed the title ~~1、solve the problem of failed parsing of STL containers~~ solve the problem of failed parsing of STL containers Aug 23, 2024

HenryRiley0 force-pushed the stl1 branch from 188c4bd to 0413cb6 Compare August 23, 2024 08:14

HenryRiley0 force-pushed the stl1 branch from 0413cb6 to 330069b Compare August 23, 2024 10:22

GitMensch marked this pull request as draft August 23, 2024 17:27

HenryRiley0 force-pushed the stl1 branch from a0d3c41 to ece8d98 Compare August 26, 2024 06:19

HenryRiley0 marked this pull request as ready for review August 26, 2024 06:21

HenryRiley0 force-pushed the stl1 branch from fd6e84b to 4c51a66 Compare September 13, 2024 09:45

HenryRiley0 force-pushed the stl1 branch from 436cd8c to 5c1fbd9 Compare September 18, 2024 09:43

GitMensch requested changes Sep 18, 2024

View reviewed changes

src/backend/gdb_expansion.ts Outdated Show resolved Hide resolved

src/backend/gdb_expansion.ts Outdated Show resolved Hide resolved

src/mibase.ts Outdated Show resolved Hide resolved

HenryRiley0 force-pushed the stl1 branch from 5c1fbd9 to 6a658d7 Compare September 19, 2024 07:28

HenryRiley0 requested a review from GitMensch September 19, 2024 07:32

HenryRiley0 marked this pull request as draft September 19, 2024 08:35

HenryRiley0 force-pushed the stl1 branch from 6a658d7 to 456d43b Compare September 19, 2024 08:59

HenryRiley0 marked this pull request as ready for review September 19, 2024 09:00

GitMensch reviewed Sep 19, 2024

View reviewed changes

solve the problem of failed parsing of std containers, add test suite

1328784

HenryRiley0 force-pushed the stl1 branch from 456d43b to 1328784 Compare September 19, 2024 09:38

HenryRiley0 requested a review from GitMensch September 19, 2024 09:39

GitMensch merged commit 33bbedd into WebFreak001:master Sep 19, 2024
5 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

solve the problem of failed parsing of STL containers #433

solve the problem of failed parsing of STL containers #433

HenryRiley0 commented Jul 22, 2024

GitMensch commented Aug 22, 2024

HenryRiley0 commented Aug 23, 2024 •

edited

Loading

HenryRiley0 commented Aug 23, 2024

GitMensch commented Aug 23, 2024

GitMensch commented Aug 23, 2024

HenryRiley0 commented Aug 23, 2024

GitMensch commented Aug 23, 2024

HenryRiley0 commented Aug 23, 2024

HenryRiley0 commented Aug 23, 2024

HenryRiley0 commented Aug 26, 2024

GitMensch commented Aug 26, 2024

HenryRiley0 commented Aug 27, 2024 •

edited

Loading

GitMensch commented Sep 13, 2024

HenryRiley0 commented Sep 14, 2024

GitMensch commented Sep 18, 2024

HenryRiley0 commented Sep 18, 2024

GitMensch left a comment

GitMensch left a comment

GitMensch Sep 19, 2024

HenryRiley0 Sep 19, 2024 •

edited

Loading

GitMensch commented Sep 19, 2024

solve the problem of failed parsing of STL containers #433

solve the problem of failed parsing of STL containers #433

Conversation

HenryRiley0 commented Jul 22, 2024

GitMensch commented Aug 22, 2024

HenryRiley0 commented Aug 23, 2024 • edited Loading

HenryRiley0 commented Aug 23, 2024

GitMensch commented Aug 23, 2024

GitMensch commented Aug 23, 2024

HenryRiley0 commented Aug 23, 2024

GitMensch commented Aug 23, 2024

HenryRiley0 commented Aug 23, 2024

HenryRiley0 commented Aug 23, 2024

HenryRiley0 commented Aug 26, 2024

GitMensch commented Aug 26, 2024

HenryRiley0 commented Aug 27, 2024 • edited Loading

GitMensch commented Sep 13, 2024

HenryRiley0 commented Sep 14, 2024

GitMensch commented Sep 18, 2024

HenryRiley0 commented Sep 18, 2024

GitMensch left a comment

Choose a reason for hiding this comment

GitMensch left a comment

Choose a reason for hiding this comment

GitMensch Sep 19, 2024

Choose a reason for hiding this comment

HenryRiley0 Sep 19, 2024 • edited Loading

Choose a reason for hiding this comment

GitMensch commented Sep 19, 2024

HenryRiley0 commented Aug 23, 2024 •

edited

Loading

HenryRiley0 commented Aug 27, 2024 •

edited

Loading

HenryRiley0 Sep 19, 2024 •

edited

Loading