Fermin's raw notes

Fermín's raw notes on September 27th 2018 audio

• Current functionality in the reviewed PR covers basically entity creation plus retrieval of context documents (including several levels of indirection?)
• A review of the functional tests (news and old) is done. There wasn't time to review the code parts :(
• A cache of context is hold in memory (not sure of having understanding this correctly... I could be wrong)
• Types and attributes in NGSI are identified by URI+alias. URI can be a URL or a URN.
• Entity IDs are URI
• URI syntax is check by the own code (i.e. no extra lib for that)
• Concerns about how the current data model ("entities" collection in MongoDB) is used. The most important concerns are: 1) alias should be used in the key-map to align with NGSIv2, 2) context URL should be added as an extra field, 3) metadata should not be overloaded to store alias (would break NGSIv2 compatibility). It is important to define a good data model, otherwise interoperability scenarios will not be possible.
• One of the function tests (0000_ngsild/ngsild_url_parse.test) is a good summary of all the URL requests patterns that JSON-LD support. Note that some pieces of the URL can be URL in sequence.
• Fermín thinks that the URL routing currently implemented in Orion could cope with the above case with small changes. He will try to confirm/discard it with a real test in the code.
• The are modifications in existing .test but seems to be minor things that, eventually, will be re-aligned with the existing .test if this is merged sometime into Orion main code. Similar changes to align in the functional .test framework (testHarness.sh / harnessFunctions.sh).
• Part of the unit tests (the ones which "bother") has been disabled.

Fermín's raw notes on October 2nd 2018 audio

• This time we focus on code parts of Orion LD PR #2
• Some doubts on how URL coding/encoding works in Orion utilization scenarios. We need to check and do a test with a) URLs with slashes, b) URL with coded slashed, and see how it looks like at: 1) what curl sends (using nc as receiver), 2) tcpdump on the wire, 3) first entry point in Orion (i.e. microhttpd callback)
• Duplicated stuff: orion main program, base CMakeList.txt, RPM-stuff. Should be easy to “join” with the original ones?
• PR #2 include some changes done previously to start the work in Orion-LD but not merged in Orion main. The mix is sometimes difficult to distinguish. I mean the changes in the PR (closed without merge): https://github.com/apinf/fiware-orion/pull/35
• Code in general is full of C patterns (char* everywhere, direct memory management, etc.).
• Simplification in URL routes, coupled with the new URL parsing and routing strategy. However, after new feedback provided by Cantera, maybe the new URL parsing and routing strategy is wrong.
• New libraries under lib/orionld
• Library /common
  • Self-made kjson library for JSON parsing, storing & rendering. High level of coupling.
  • The same can be achieved using existing library (rapidjson) but the reason for not using it is that Ken found it difficult to use.
  • Context JSON download using librcurl
  • urlCheck logic may be duplicated. Why? Ken doesn’t remember
• Library /context
  • Context download, storing & lookup
  • Dynamic linked list, each node being the URL (from where it has been downloaded) and a tree (representing the JSON). The tree is in sequence a list of key-values. Values are in sequence URLs or a JSON object in the following pattern: {@id: url, @type: url}
  • Thus, the context trees managed by this module are not arbitrary trees. Indeed, they have a very fixed structure which could be easily modeled in a class/struct.
  • No parsing of the JSON to internal classes/structs. JSON tree is used directly in memory.
• Library / rest:
  • Based in checksum calculations + string comparison to avoid false positives
• Library /serviceRoutine:
  • Only post entities operation in this PR
  • Separate serviceRoutine from the regular (i.e. NGSIv2) entity creation serviceRoutine. Thus, logic associated to regular entity creation (e.g. forwarding) is not executed.
  • A lot of logic from serviceRoutine entry point to the mongoBackend invocation. MonboBackend invocation method is the regular one.
  • Doesn’t use internal structures (such as the NGSI classes). JSON tree from payload is used directly. Not seen in this PR but the JSON tree is also rendered directly (in get operations)