Prolog Analysis step duration

[YamamotoKaderate]

I analyzed a 2.8Mb executable and produced an 500k lines/facts prolog file
The JSON generation step (edit: prolog analysis step) is running for 70h.
Is there a way to estimate the remaining duration of this step?

[sei-ccohen]

We often use this command:

grep "Entering stage" prolog-log-file

The stages after "Initial reasoning complete" (which is usually reached very quickly) are:

Entering stage 'Initial reasoning complete'.
Entering stage guessVirtualFunctionCall.
Entering stage guessVFTable.
Entering stage guessVBTable.
Entering stage guessDerivedClass.
Entering stage guessMethod.
Entering stage guessVFTableEntry.
Entering stage guessDeletingDestructor.
Entering stage guessRealDestructor.
Entering stage guessNOTConstructor.
Entering stage guessConstructor.
Entering stage guessNOTMergeClasses.
Entering stage guessClassHasNoBase.
Entering stage guessUnlikelyConstructor.
Entering stage guessMergeClassesB.
Entering stage guessMergeClassesC1.
Entering stage guessMergeClassesC2.
Entering stage guessMergeClassesC3.
Entering stage guessMergeClassesC4.
Entering stage guessMergeClassesG.
Entering stage guessFinalDeletingDestructor.
Entering stage guessLateMergeClassesF1.
Entering stage guessLateMergeClassesF2.
Entering stage guessLateMergeClasses_G1.
Entering stage guessLateMergeClassesG.
Entering stage guessCommitClassHasNoBase.
Entering stage reportResults.

Although the time between steps is often not linear. Slow downs in the MergeClasses stages are pretty common for large programs because there tends to be N-squared pairs of methods to consider merging.

We're aware that the performance is less than optimal. We're working with the SWI Prolog developers to develop a quicker approach.

[sei-ccohen]

Oh, and I assumed you meant the Prolog analysis step, not the JSON generation step (most people run those together). If it's really in the JSON generation step, then there's something seriously wrong, since that step usually only takes a few seconds to a few minutes. If that's really the step with the problem, you should probably email us a link to your results file so that we can investigate further.

[YamamotoKaderate]

My bad, Prolog analysis indeed. Thank you for your quick response

[sei-eschwartz]

You may also want to take a look at these notes if you have not done so already.

[YamamotoKaderate]

Yeah, I read the notes. It might be interesting to document this stages.
Well, at least I'm not stuck. Supposing line numbers is a good indicator of time spent
wc -l prolog-log-file
6567479
grep -n "Entering stage" prolog-log-file
1:Entering stage 'Loading data'.
46:Entering stage 'Initial reasoning'.
49:Entering stage concludeMethod.
4988:Entering stage concludeVFTableOverwrite.
4989:Entering stage concludeConstructor.
4990:Entering stage concludeNOTConstructor.
11109:Entering stage concludeVFTable.
11151:Entering stage concludeVFTableWrite.
11292:Entering stage concludeVFTableEntry.
11320:Entering stage concludeNOTVFTableEntry.
11326:Entering stage concludeVFTableSizeGTE.
11355:Entering stage concludeVFTableSizeLTE.
11364:Entering stage concludeVBTable.
11365:Entering stage concludeVBTableWrite.
11366:Entering stage concludeVBTableEntry.
11367:Entering stage concludeObjectInObject.
11368:Entering stage concludeDerivedClass.
11381:Entering stage concludeNOTDerivedClass.
11382:Entering stage concludeEmbeddedObject.
11383:Entering stage concludeNOTEmbeddedObject.
11388:Entering stage concludeDeletingDestructor.
11389:Entering stage concludeRealDestructor.
11390:Entering stage concludeNOTRealDestructor.
17414:Entering stage concludeClassSizeGTE.
57740:Entering stage concludeClassSizeLTE.
57741:Entering stage concludeClassHasNoBase.
57753:Entering stage concludeClassHasUnknownBase.
57760:Entering stage concludeReusedImplementation.
57761:Entering stage concludeNOTMergeClasses.
57930:Entering stage concludeTrigger.
79087:Entering stage concludeMergeVFTables.
79097:Entering stage concludeMergeClasses.
79125:Entering stage concludeClassCallsMethod.
191852:Entering stage 'Initial reasoning complete'.
191856:Entering stage guessVirtualFunctionCall.
251931:Entering stage guessVFTable.
299433:Entering stage guessVBTable.
300205:Entering stage guessDerivedClass.
300225:Entering stage guessMethod.
370147:Entering stage guessVFTableEntry.
1027245:Entering stage guessDeletingDestructor.
1027282:Entering stage guessRealDestructor.
1027283:Entering stage guessConstructor.
1542168:Entering stage guessNOTMergeClasses.
1603781:Entering stage guessClassHasNoBase.
1971730:Entering stage guessUnlikelyConstructor.
1972377:Entering stage guessMergeClassesB.
2742171:Entering stage guessMergeClassesC1.
3408624:Entering stage guessMergeClassesC2.
3416316:Entering stage guessMergeClassesC3.
3422363:Entering stage guessMergeClassesC4.
5801060:Entering stage guessMergeClassesG.
5894825:Entering stage guessFinalDeletingDestructor.
5894826:Entering stage guessLateMergeClassesF1.
6322769:Entering stage guessLateMergeClassesF2.
6322770:Entering stage guessLateMergeClassesG.

[sei-ccohen]

It would seem that you're quite close to the end in fact. My guess would be that you'll finish analysis in a couple of hours at most. Explaining the flow with the Prolog analysis is quite complicated, and characterizing the nature of the performance problems is even harder. You can get some hints by looking at the rules in guess.pl that enit the messages about the various stages, but I wouldn't expect it to mean too much...