Skip to content

Loading of aag file consumes massive memory resources and never ends #122

Closed
@Hemer911

Description

@Hemer911

Hi,

I'm trying to perform a simple load of aag file (code below).
The load function never ends and cause massive memory usage (I stopped the run at 12GB).

I've noticed that the code get stuck at the loop in line 262 of parser.py.
I think that the reason is that at lines 273, 274 the parser aggregates boolean expressions from all the nodes parsed so far. It causes a very long boolean expressions and a massive memory usage.

Can you please assist?

Some technical details:
i10.aig is an example database I got from ABC tool.
I translated it to aag using aigtoaig from AIGER package.

Here is the aag header:
aag 2932 257 0 224 2675

Here is my code:

  1. import aiger
  2. databasePath = "./"
  3. fileName = "i10.aag"
  4. aig1 = aiger.load(databasePath+fileName)

Activity

mvcisback

mvcisback commented on Mar 2, 2021

@mvcisback
Owner

Hi @Hemer911

Could you please attach the file so I can reproduce.

Hemer911

Hemer911 commented on Mar 2, 2021

@Hemer911
Author

i10.aag.gz
Thanks

mvcisback

mvcisback commented on Mar 2, 2021

@mvcisback
Owner

@Hemer911 I just published a patched py-aiger version.

I now build the AIG directly rather than using the BoolExpr DSL.

In retrospect, its not surprising given that the current composition code is worst case linear is the right circuit (and thus quadratic for repeated composition of a growing circuit).

I plan to eventually fix this, but its a bit subtle to do and keep things immutable.

Anyway, if you could confirm that it works for you, that'd be great!

Hemer911

Hemer911 commented on Mar 3, 2021

@Hemer911
Author

Hi,
I tried the new version with the file above and 2 other examples and it works great.
Thanks a lot

Hemer911

Hemer911 commented on Mar 3, 2021

@Hemer911
Author

One thing tough, what do you think would be the upper limit of the load function (in terms of number of nodes in the loaded graph)? 10K? 100K? more?

mvcisback

mvcisback commented on Mar 3, 2021

@mvcisback
Owner

Hard to say. I think, so long as everything fits in ram (load the file as a string - would be fairly straightforward to change), it should work.

The current algorithm does a linear pass through the aag to build the graph and then another topologically ordered pass to build the circuit.

I think it should grow ~linearly, but there's always surprises...

That said, I think the bigger problem with big circuits at the moment is manipulation, which due to immutability copies the circuit.

There's a lazy api I've been working on which helps amortize the cost, but its not fully baked. I've used it to unroll sequential circuits, and there it worked for large (~10-100k) circuits.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @mvcisback@Hemer911

        Issue actions

          Loading of aag file consumes massive memory resources and never ends · Issue #122 · mvcisback/py-aiger