TODO

Things to do for piuparts
=========================

Please also check the BTS, especially the bugs with a severity higher than
wishlist! And see the bottom of this file for wishlist bugs someone once
expressed…


Some stuff we might tackle during the bullseye development cycle:

- switch to python3 (that's actually a must)

- README_server.txt: rewrite style a bit more. its super easy to setup now!

- ==== piuparts.debian.org specific configuration
 ^^^^^^^ scripts should be the headline, not piu.d.o

- move more bits from README_pejacevic.txt to README_server.txt

- split out README_protocol and README_piuparts.conf? -> in piuparts.conf manpage maybe?

- support multiple architectures: #762308
  - piuparts-report should have a list of available arch and list packages
    only available on untested archs in a new state
    "(depends-)not-available-on-tested-archs"
  - master should (per default) only schedule packages which are not available
    on the master arch to slaves of different archs ->
    "schedule-evenly-to-slaves = no"

- more stats and graphs:
  - new section stats page:
    - packages processed per day and section, master writes submissions.txt
      since 0.45 for all sections.
    - generate simple diagrams: number of source + binary packages in all
      single distros: lenny, squeeze, wheezy, jessie, stretch, buster, sid.
    - graph about piuparts stati for all sections combined? (possible ignore
      successful)

- master should create the master and backup directories, if they don't exist.
  If master does that remove that sentence from README_server.txt again,
  same with slave and tmp.

- if it weren't for 'slave-bin/slave_cleanup', the slave would only need
  rights to run "sudo piuparts" but nothing else. If we can clean this up,
  the sudoers.d should recommend sudo (lsof|kill|umount) for admins.

- if there were real schroot support, piuparts could be used without sudo.
  (#708663)

- use network namespaces to disable network during the tests:
  - < weasel> says: unshare --uts --ipc --net -- sh -c 'ip addr add 127.0.0.1/8 dev lo && ip link set dev lo up && stuff'
    and points to https://anonscm.debian.org/gitweb/?p=mirror/dsa-puppet.git;a=blob;f=modules/porterbox/files/dd-schroot-cmd#l104
  - problem might be access to the mirror, either (bind mounted) nfs access will still work in the chroots or do as its done on the porterboxes:
    apt-get install -d <foo>, unshare apt-get install foo

- add a sample config with all possible keys set to some useful value
  (like /usr/share/doc/apt/examples/configure-index.gz)

- generate piuparts.1.txt automatically from piuparts.py - see this blog post
  for a nice howto:
  http://andialbrecht.wordpress.com/2009/03/17/creating-a-man-page-with-distutils-and-optparse/
  - though this seems pretty complicated... maybe rather grep for
    parser.add_option and help= in piuparts.py ?!
    - requires merging all the additional infomation in piuparts.1.txt into
      piuarts.py
    - parsing piuparts --help output may be easier than parsing piuparts.py
  - elso: examples are duplicated in piuparts.1.txt and README.txt

- check the logfiles (especially pass/) for
  - "Exception in thread"
  - java stacktraces
  - "Can't locate .* in @INC"

- we should probably have an install test with --enable-recommends and
  without --warn-on-others to avoid adding artificial barriers where package
  subsets are configured
  (wheezy2jessie-rcmd is *not* the solution for this)

- report actually ignored files/patterns ROT13 encoded to be able to spot and
  reschedule such tests

- p-r: in the section summary page, report the piuparts flags being used

- p-s: report age of the basetgz being used

- solve the discrepancies between "dist-upgrade" (apt) and "distupgrade"
  (piuparts), adopting apt's naming


Ideas for later:

- maybe compress all logfiles

- install from git/Makefile: remove the need for /etc/piuparts

- accept a PIUPARTS_CONF environment variable everywhere to point to a different
  piuparts.conf

- write reportbug-like wrapper for mass bug filing (start simple, make it more
  sophisticated later).

- rework known_problems:
  - use a number prefix for sorting
  - add title information
  - piuparts-report: "discover" the available known_problems, don't hardcode the
    list
  - drop _issue/_error duplication, have flags inside to indicate whether to
    generate _issues.tpl (pass/) and/or _error.tpl (fail/ bugged/ affected/)
  - rework known problems to a python-friendlier format

- the templates used by piuparts-report.py should all be taken from
  /etc/piuparts/templates/ and not be included in the python source

- a redirect of http://piuparts.d.o/foo to http://p.d.o/source/f/foo.html would
  be nice


More ideas for even later:

- make it possible to call aptitude (or similar) instead of apt-get and allow to
  override the commandline arguments.

- "decorate" (strike-through) bug links generated by piuparts-analyze to
  indicate resolved state (take package version into account!)

- report:
  - write stats about the reasons for failures, as its done with shell scripts
    now (piuparts-analyze.py is an existing "fragment".)
  - RSS feeds of logs
  - do more fancy R graphs, eg. also per state
  - link (and target) to piuparts.d.o configuration is static to pejacevic. should
    refer to the actual hosts configuration if running somewhere else

- not sure if it's a sensible thing to to, but provide a way to turn off
  debugging output for piuparts.py - see
  http://docs.python.org/library/logging.html

- automated testing of piuparts using an archive of known broken packages:
  - create archive of broken packages to provide test cases for piuparts testing.
  - create emacspeak-broken-dpkg-preconfigure package for broken repo. (then later
    put more broken packages in there and use that for testing piuparts)



(Closed) wishlist bugs are listed here so it's easier to find information
about them should one wants to implement support for those. Usually wishlist
bugs are closed not because the feature is unwanted, but because noone is
working on them… see http://blog.liw.fi/posts/wishlist-bugs for a longer
rationale.

piuparts:
- #664727 - support cowbuilder in addition to pbuilder
- #798306 - piuparts should support [trusted=yes] in "file deb src" sources.list lines
- #588042 - add an option to override the log file instead of append to it
- #354842 - new option: remove /usr/local
- #558819 - new option: check that daemon stop scripts don't fail even if the daemon doesn't run
- #660896 - new option: use etckeeper and show how modified files have changed
- #403648 - new test: Try unpack, then remove?
- #561444 - new test: install, remove configuration files, re-install, check that configuration files don't re-appear
- #561445 - new test: install, mangle configuration file, re-install, check that manglement is still there
- #665016 - new test: more checks on alternatives
- #810323 - new test: deconfiguring and configuring a package again
- #658731 - add pattern for owned files that are always an error
- #682741 - add a second set of ignored files and patterns for "partial purge" tests

piuparts-report:
- #685813 - add a single report of all failures (from all sections) ordered by age