Add real-life examples of Python RPM #3177

FrostyX · 2024-06-21T10:03:08Z

FrostyX · 2024-06-21T10:05:52Z

python/examples/installing-a-package.py

+    filename = os.path.basename(path)
+    hdr = ts.hdrFromFdno(fp.fileno())
+    ts.addInstall(hdr, filename, "i")
+ts.run(lambda *args: print(args), {})


I have never actually installed anything through the Python API in real life, so I am not sure what the ts.run arguments should be. Same for the erasing example above.

Also, this script fails with

# FIXME this snippet fails with TypeError: 'NoneType' object cannot be interpreted as an integer FATAL ERROR: python callback ??? failed, aborting!

Can you please help me finish this example?

You'll need to supply a callback that handles at least file open and close (rpm.RPMCALLBACK_INST_OPEN_FILE and rpm.RPMCALLBACK_INST_CLOSE_FILE), based on the key you supply. It's all ... rather arcane.

See "Callbacks" and the following slides in https://web.archive.org/web/20050320013335/http://www.ukuug.org/events/linux2004/programme/paper-PNasrat-1/rpm-python-slides/frames.html

That said, the callback example actually sets a rather bad example: you don't want to use a header in the key, because it wastes enormous amounts of memory. If you want output a NEVRA in the callback, pass that string instead.

FrostyX · 2024-06-21T10:06:09Z

python/examples/querying-a-single-package-more-info.py

+        f"Group       : {hdr[rpm.RPMTAG_GROUP]}\n"
+        f"Size        : {hdr[rpm.RPMTAG_SIZE]}\n"
+        f"License     : {hdr[rpm.RPMTAG_LICENSE]}\n"
+        f"Signature   : TODO\n"


What is the correct tag here, please?

The answer is not a single tag, it's either RPMTAG_RSAHEADER or RPMTAG_DSAHEADER (will cover both original DSA and ECDSA)

FWIW, most people prefer the shorter hdr['license'] syntax, but of course using rpm.RPMTAG_* symbols, errors are more up front.

FrostyX · 2024-06-21T10:13:08Z

All of the proposed examples show how to implement various rpm commands. I don't know whether it would be worth it having them in a separate (sub)directory or not.

pmatilai · 2024-06-24T09:35:00Z

python/examples/querying-a-single-package.py

+ts = rpm.TransactionSet()
+mi = ts.dbMatch("name", name)
+hdr = next(mi, None)
+if hdr:


This should be in a loop just like in your querying-all-installed version - there can always be more than one package with a given name, and callers better handle that correctly.

Looping over the return value (for hdr in mi:) also has the advantage that you can skip the next() call and the if hdr check; if an empty list is returned it will simply cause the loop the be skipped.

~~(You can also check for an empty list after the loop, e.g. this will print "No results" only when the list is empty):~~

~~mi = ts.dbMatch("name", name) for hdr in mi: ... if not mi: print("No results")~~

Actually, I take that back. Because mi is an iterator, not a list, it will become False when exhausted, so that code will always print "No results" at the end of the loop. My mistake.

pmatilai · 2024-06-24T09:38:24Z

python/examples/querying-a-single-package-more-info.py

+ts = rpm.TransactionSet()
+mi = ts.dbMatch("name", name)
+hdr = next(mi, None)
+if hdr:


This should be a loop to handle multiple identically named packages.

pmatilai · 2024-06-24T09:40:20Z

python/examples/searching-for-packages.py

+
+ts = rpm.TransactionSet()
+mi = ts.dbMatch()
+mi.pattern("name", rpm.RPMMIRE_GLOB, "kernel*")


I would make the package name in these examples something you pass on the command line to make them actually usable on real systems which are unlikely to have "hello" package installed.

pmatilai · 2024-06-24T09:49:19Z

python/examples/querying-all-installed-packages.py

+
+ts = rpm.TransactionSet()
+mi = ts.dbMatch()
+for hdr in mi:


This isn't wrong in such a small example of course, but assigning the match iterator to a variable can leave the database cursor open for much longer than intended. With read-cursors it's not a big deal with in the post BerkeleyDB world, but when you don't actually need the match iterator handle outside the loop (ie the usual case), I'd recommend this style instead:

for hdr in ts.dbMatch(...): # do stuff

Similar issues exist with the 'ts' handle of course - doesn't matter at all in such small examples but people commonly don't realize that the 'ts' handle will also keep the database open for much longer than intended. But, in typical usage the same ts handle is used in multiple operations and in that case it does make sense to assign it to a variable, whereas the match iterator is mostly useless once looped through once.

pmatilai · 2024-06-24T13:20:18Z

All of the proposed examples show how to implement various rpm commands. I don't know whether it would be worth it having them in a separate (sub)directory or not.

Anything you do with the bindings is going to resemble something rpm does, I don't see these being any different from the pre-existing examples in that, and I doubt we'll have too many examples to reasonably fit into a single directory anytime soon.

One observation here is that these names are awfully verbose in rpm, the land of the terse, and look a bit odd because of that. So maybe some shorter names would do 😄 Something like 'rpm-qa.py' seems like equally telling to me (but maybe not to someone less familiar with rpm, so dunno)

ferdnyc · 2024-07-29T23:02:21Z

If "real-world" examples are the goal, mightn't it be better to mine actual real-world code that uses the Python API for examples? DNF comes to mind, as a blindingly-obvious source of existing, working code based on the RPM Python bindings. 😁

(The examples can be simplified/sanitized for clarity, of course, but they'll be far more "real-world" if they're based on... well... real code.)

pmatilai · 2024-07-30T06:40:04Z

Dnf is not the best of sources for rpm API usage. It does most of its work on the libsolv level rather than rpm, and the little rpm API usage there is suffers from the yum heritage.

FrostyX · 2024-07-30T17:21:02Z

Thank you all for so many comments. I am sorry for being inactive on this PR for a while. I was partially on vacation and now trying to catch up. I will update the PR soon.

FrostyX · 2024-08-03T17:42:53Z

Once again, sorry for the delay.
@pmatilai I think I addressed all your comments. Can you please take a look again?

If "real-world" examples are the goal, mightn't it be better to mine actual real-world code that uses the Python API for examples?

I am not aware of any reasonably well-known higher-level tools that use RPM Python API to query, install, and erase packages. I know a few minor tools (some of them mine) that sprinkle some RPM Python API usage here and there but nothing that I'd safely recommend as an example.

It may be a good idea to link such tools in the documentation (Let me know your stance on this, I will be submitting a documentation PR once this one gets merged).

@pmatilai already addressed DNF and its libsolv usage but I'd maybe disagree in general.

I went for example Python implementations of commonly used rpm commands because they are quite easy to understand, it is IMHO a reasonable expectation that Python RPM API users know rpm commands, and also devs will likely compare values they get from the API with whatever rpm prints. Or see some rpm output and say to themself "how the heck do I get this value via API?". At least I do that often.

In the documentation, I can use a different terminology than "real-world" if it is deceiving :-)

pmatilai · 2024-08-06T08:42:29Z

python/examples/rpm-e.py

+ts = rpm.TransactionSet()
+for name in sys.argv[1:]:
+    ts.addErase(name)
+ts.run(lambda *args: ..., "")


A transaction with nothing but erase elements probably does happen to work without a callback. But one should always supply a callback to the transaction, especially these being "official" examples.

Maybe demonstrate some other callback events for this? Or combine with the install example, because we don't want people to think one can only do one kind of operation in a transaction even if the rpm cli limits you that way.

But one should always supply a callback to the transaction, especially these being "official" examples.

I agree. And even though the lambda works there, it's quite WTF-looking.
Updated, PTAL

Maybe demonstrate some other callback events for this? Or combine with the install example, because we don't want people to think one can only do one kind of operation in a transaction even if the rpm cli limits you that way.

This is what I don't want to do. My plan was to explain this in the documentation page, that it is possible to combine different operations within one transaction. If you want to, we can add some python example showing it.

But I would like to keep this example as simple as possible, showing just how to do rpm -e hello

I just think it gives you a rather strange impression of the API when it's demonstrated as two separate cases with two entirely different callbacks. But okay, that can be resolved with two simple comments, something like below, respectively for erase and install examples:

# erasure-only transactions don't strictly require anything from the callback # most operations require at least rpm.RPMCALLBACK_INST_OPEN_FILE and rpm.RPMCALLBACK_INST_CLOSE_FILE to be handled

I put the comment into the rpm-e.py callback

pmatilai · 2024-08-22T10:11:20Z

python/examples/rpm-i.py

+for path in sys.argv[1:]:
+    with open(path, "r") as fp:
+        hdr = ts.hdrFromFdno(fp.fileno())
+        ts.addInstall(hdr, (hdr.nvr, path), "i")


You'll want to use "u" in there (making this the equivalent of 'rpm -U' instead). -i is rather special case operation that is best left to people who know to look for it. Sorry for missing this earlier.

Good catch, thank you. Updated.

See rpm-software-management/rpm-web#54

pmatilai · 2024-08-26T05:54:30Z

Okay fair enough, I think we've reviewed and bikeshedded this enough by now 😄
Thanks for the examples!

FrostyX · 2024-08-26T08:31:05Z

Thank you very much for the thorough review, I learned a lot of new things through it.
I am planning to follow this PR up with changes to the RPM Language Bindings on https://rpm.org/documentation.html

FrostyX commented Jun 21, 2024

View reviewed changes

FrostyX force-pushed the python-examples branch from b4e0bde to c073851 Compare June 21, 2024 10:09

FrostyX mentioned this pull request Jun 21, 2024

Add real-life examples of Python RPM rpm-software-management/rpm-web#54

Closed

pmatilai reviewed Jun 24, 2024

View reviewed changes

ffesti self-assigned this Jul 3, 2024

ffesti added the documentation label Jul 3, 2024

FrostyX force-pushed the python-examples branch from c073851 to 852582e Compare August 3, 2024 17:25

pmatilai reviewed Aug 6, 2024

View reviewed changes

FrostyX force-pushed the python-examples branch from 852582e to 04f5156 Compare August 7, 2024 10:58

pmatilai reviewed Aug 22, 2024

View reviewed changes

Add real-life examples of Python RPM

019fad7

See rpm-software-management/rpm-web#54

FrostyX force-pushed the python-examples branch from 04f5156 to 019fad7 Compare August 22, 2024 13:54

pmatilai merged commit e927be7 into rpm-software-management:master Aug 26, 2024
1 check passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add real-life examples of Python RPM #3177

Add real-life examples of Python RPM #3177

FrostyX commented Jun 21, 2024

FrostyX Jun 21, 2024

pmatilai Jun 24, 2024

pmatilai Jun 24, 2024

FrostyX Jun 21, 2024

pmatilai Jun 24, 2024 •

edited

Loading

FrostyX commented Jun 21, 2024

pmatilai Jun 24, 2024

ferdnyc Jul 29, 2024

ferdnyc Jul 29, 2024 •

edited

Loading

ferdnyc Jul 29, 2024

pmatilai Jun 24, 2024

pmatilai Jun 24, 2024

pmatilai Jun 24, 2024

pmatilai Jun 24, 2024 •

edited

Loading

pmatilai commented Jun 24, 2024 •

edited

Loading

ferdnyc commented Jul 29, 2024

pmatilai commented Jul 30, 2024

FrostyX commented Jul 30, 2024

FrostyX commented Aug 3, 2024

pmatilai Aug 6, 2024 •

edited

Loading

FrostyX Aug 7, 2024

pmatilai Aug 22, 2024

FrostyX Aug 22, 2024

pmatilai Aug 22, 2024

FrostyX Aug 22, 2024

pmatilai commented Aug 26, 2024

FrostyX commented Aug 26, 2024

Add real-life examples of Python RPM #3177

Add real-life examples of Python RPM #3177

Conversation

FrostyX commented Jun 21, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pmatilai Jun 24, 2024 • edited Loading

Choose a reason for hiding this comment

FrostyX commented Jun 21, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ferdnyc Jul 29, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pmatilai Jun 24, 2024 • edited Loading

Choose a reason for hiding this comment

pmatilai commented Jun 24, 2024 • edited Loading

ferdnyc commented Jul 29, 2024

pmatilai commented Jul 30, 2024

FrostyX commented Jul 30, 2024

FrostyX commented Aug 3, 2024

pmatilai Aug 6, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pmatilai commented Aug 26, 2024

FrostyX commented Aug 26, 2024

pmatilai Jun 24, 2024 •

edited

Loading

ferdnyc Jul 29, 2024 •

edited

Loading

pmatilai Jun 24, 2024 •

edited

Loading

pmatilai commented Jun 24, 2024 •

edited

Loading

pmatilai Aug 6, 2024 •

edited

Loading