make dump function true json

[Rehouma63]

No description provided.

[petersilva]

I run a flow test, then do an sr3 dump. when I load the output into jslint.com, I can see that
the brackets are unbalanced. Each component is nested in the component above it in the list.
so there is a missing close brace, per component.

Then it ends with:

Error: Parse error on line 908:
...]								}								"sarra": {									
---------------------^
Expecting 'EOF', '}', ',', ']', got 'STRING'

[petersilva]

oh... and I skipped the *dumping: " title in my copy/paste.

[Rehouma63]

I run a flow test, then do an sr3 dump. when I load the output into jslint.com, I can see that the brackets are unbalanced. Each component is nested in the component above it in the list. so there is a missing close brace, per component.

Then it ends with:

Error: Parse error on line 908:
...]								}								"sarra": {									
---------------------^
Expecting 'EOF', '}', ',', ']', got 'STRING'

that's weird, i used another json validator and it was valid, ill take a deeper look.

[petersilva]

jsonlint still reports similar issues:

Still indenting by 1 per component (cpost, cpump, flow, poll, ... )
right now, the config structure is the parent of cpost (correct) but cpump is part of cpost, flow is part of cpost... etc subsequent component inside the previous one, instead of being the next item under config.

I also see the same error at the end.

my method of validation, is to use the dynamic_flow test from sr_insects.
I do ./flow_setup.sh

to start it running, then sr3 dump >kk and then copy/past the text into jsonlint.com

A second method, I remove the "dumping: " from the beginning of the file, and then
try to load it in python:

fractal% python3
Python 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import json
>>> f=open("kk","r")
>>> data=f.read()
>>> j=json.loads(data)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.10/json/decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 2646 column 1 (char 115313)
>>> 

If you have a different method of validating the json, please describe.

[petersilva]

You can see in jsonlint that all subsequent parts of the dump are offset/nested after "Configs".

[Rehouma63]

jsonlint validates the format but gives an error about duplicate key, not sure if it should be adressed. https://jsonformatter.curiousconcept.com/# is valid, https://codebeautify.org/jsonvalidator too.

[Rehouma63]

I used your method with python3, seems to be working...

import json
f=open("output","r")
data= f.read()
j = json.loads(data)

[petersilva]

nope... for every configuration I test, it fails. what test data are you using? Are you starting a config? are there any configurations running, or configured when you run the dump?

[Rehouma63]

I runned the dynamic flow tests (all 36 tests passed) then i do sr3 dump. Configurations are running when I do sr3 dump:

Maybe there's something wrong with my config ? Here's the output file: DumpOutput.txt

[petersilva]

So it really works now. Great!... it looks a little clumsy with the enumerate mechanism, I had a look at the "Processes" loop and found I could reduce it from four lines to one in idiomatic python.

The other cases do not look as simple, but are perhaps amenable to similar improvement. But it works... working beats beautiful every time.

[petersilva]

I tried to merge it, but the merge does not work. I'm not sure why... I might have broken it by trying to rebase. It might be better to start over with a fresh branch, and make one commit with the correct final version in it.... Yeah, I think that is the best way to go about it, you just copy/paste the final version of the dump routine into a fresh branch off of v03_wip (say v03_issue634_2?), and we will save a lot of patches. It will be one clean commit with just the final result.

[petersilva]

replaced by #749