Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

awk detection is broken on Musl-based Linux distros #132

Closed
sskras opened this issue Mar 25, 2024 · 10 comments
Closed

awk detection is broken on Musl-based Linux distros #132

sskras opened this issue Mar 25, 2024 · 10 comments

Comments

@sskras
Copy link

sskras commented Mar 25, 2024

Forking the issue #130 (comment).

Currently I test only the OpenMPTCProuter distro, but OpenWRT probably suffers in the same way.

The distro provides BusyBox version of AWK which seems to have a mix of features from nawk, gawk and probably some other implementations too: https://wiki.alpinelinux.org/wiki/Awk

My idea was that the boostrap/bootstrap script should either pick up the OS-native version (/usr/bin/awk -> ../../bin/busybox) or bootstrap its' own version (/usr/pkg/bin/nawk).

The bootstrapping seems to to ignore the former and require the latter. The problem is that the latter isn't built yet at the moment it gets required:

===> bootstrap command: bootstrap/bootstrap 
===> bootstrap started: Sun Mar 24 21:02:38 UTC 2024
===> running: /bin/sed -e 's|@DEFAULT_INSTALL_MODE@|'0755'|' /root/debug/src/github.com/NetBSD/pkgsrc.git/sysutils/install-sh/files/install-sh.in > /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh
===> running: /bin/chmod +x /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh
===> Creating default mk.conf in /root/debug/src/github.com/NetBSD/pkgsrc.git/work
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -d -o root -g root /root/debug/src/github.com/NetBSD/pkgsrc.git/work/sbin
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -d -o root -g root /root/debug/src/github.com/NetBSD/pkgsrc.git/work/share/mk
===> Bootstrapping mk-files
===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/bootstrap-mk-files/files && env CP=/bin/cp  OPSYS=Linux MK_DST=/root/debug/src/github.com/NetBSD/pkgsrc.git/work/share/mk ROOT_GROUP=root ROOT_USER=root SED=/bin/sed SYSCONFDIR=/usr/pkg/etc /bin/sh ./bootstrap.sh)
===> Bootstrapping bmake
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -d -o root -g root /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bmake
===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bmake && /bin/sh configure  --prefix=/root/debug/src/github.com/NetBSD/pkgsrc.git/work --with-default-sys-path=/root/debug/src/github.com/NetBSD/pkgsrc.git/work/share/mk --with-machine-arch=x86_64 )
===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bmake && /bin/sh make-bootstrap.sh)
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bmake/bmake /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/bmake
===> Building libnbcompat
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -d -o root -g root /root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat
===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat; /bin/sh ./configure  -C --prefix=/usr/pkg --infodir=/usr/pkg/info --mandir=/usr/pkg/man --sysconfdir=/usr/pkg/etc --enable-bsd-getopt --enable-db && /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/bmake  -j1)
===> Bootstrapping pkgtools
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -d -o root -g root /root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive
===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive; env  CONFIG_SHELL=/bin/sh /bin/sh ./configure  --enable-static --disable-shared --disable-bsdtar --disable-bsdcpio --disable-bsdunzip --disable-posix-regex-lib --disable-xattr --disable-maintainer-mode --disable-acl --without-zlib --without-bz2lib --without-iconv --without-lzma --without-lzo2 --without-lz4 --without-nettle --without-openssl --without-xml2 --without-expat --without-zstd MAKE=/root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/bmake && /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/bmake  -j1)
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -d -o root -g root /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install
===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install; env  CPPFLAGS=' -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive' LDFLAGS=' -L/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat' LIBS=' -lnbcompat' /bin/sh ./configure  -C --enable-bootstrap --prefix=/usr/pkg --sysconfdir=/usr/pkg/etc --with-pkgdbdir=/usr/pkg/pkgdb --infodir=/usr/pkg/info --mandir=/usr/pkg/man  && STATIC_LIBARCHIVE=/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/.libs/libarchive.a STATIC_LIBARCHIVE_LDADD='' PKGSRC_MACHINE_ARCH=x86_64 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/bmake  -j1)
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/add/pkg_add /root/debug/src/github.com/NetBSD/pkgsrc.git/work/sbin/pkg_add
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/admin/pkg_admin /root/debug/src/github.com/NetBSD/pkgsrc.git/work/sbin/pkg_admin
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/create/pkg_create /root/debug/src/github.com/NetBSD/pkgsrc.git/work/sbin/pkg_create
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/info/pkg_info /root/debug/src/github.com/NetBSD/pkgsrc.git/work/sbin/pkg_info
sh: -f: not found
bmake[1]: "/root/debug/src/github.com/NetBSD/pkgsrc.git/lang/gcc6/options.mk" line 77: warning: "echo  6.5.0 |   -f ../../mk/scripts/larger_symbol_version.awk" returned non-zero status
sh: -f: not found
bmake[1]: "/root/debug/src/github.com/NetBSD/pkgsrc.git/lang/gcc6/options.mk" line 77: warning: "echo  6.5.0 |   -f ../../mk/scripts/larger_symbol_version.awk" returned non-zero status
sh: {print $3; exit}: not found
bmake[1]: "/root/debug/src/github.com/NetBSD/pkgsrc.git/mk/tools/gmake.mk" line 43: warning: "/usr/bin/make --version |				  '{print $3; exit}'" returned non-zero status
===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/cwrappers && /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/bmake  MAKE_JOBS=1 PKG_COMPRESSION=none PKGSRC_KEEP_BIN_PKGS=no MAKECONF=/root/debug/src/github.com/NetBSD/pkgsrc.git/work/mk.conf install)
sh: -f: not found
bmake[1]: "/root/debug/src/github.com/NetBSD/pkgsrc.git/lang/gcc6/options.mk" line 77: warning: "echo  6.5.0 |   -f ../../mk/scripts/larger_symbol_version.awk" returned non-zero status
sh: -f: not found
bmake[1]: "/root/debug/src/github.com/NetBSD/pkgsrc.git/lang/gcc6/options.mk" line 77: warning: "echo  6.5.0 |   -f ../../mk/scripts/larger_symbol_version.awk" returned non-zero status
sh: {print $3; exit}: not found
bmake[1]: "/root/debug/src/github.com/NetBSD/pkgsrc.git/mk/tools/gmake.mk" line 43: warning: "/usr/bin/make --version |				  '{print $3; exit}'" returned non-zero status
/root/debug/src/github.com/NetBSD/pkgsrc.git/mk/pkgformat/pkg/list-dependencies: line 47: /usr/pkg/bin/nawk: not found
===> Installing dependencies for cwrappers-20220403
/root/debug/src/github.com/NetBSD/pkgsrc.git/mk/pkgformat/pkg/list-dependencies: line 47: /usr/pkg/bin/nawk: not found
*** Error code 1

Stop.
bmake: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/cwrappers
===> exited with status 1
aborted.

(I trimmed down details from all earlier steps except for the last three ones)

@sskras
Copy link
Author

sskras commented Mar 25, 2024

@jperkin responded on IRC on Sunday, March 17th, 2024:

I'd start from first-principles

firstly, does it not exist, or is /usr/pkg/bin/nawk there but somehow built with an invalid interpreter or something?

nawk is only bootstrapped if a native awk isn't found or is unsuitable, so I'd also check to see why your native awk isn't sufficient and if it can be provided/fixed instead

then if it is required and is missing, I'd be looking at the bootstrap log to figure out why it failed and how that wasn't reported as a failure to the bootstrap script

as this is Linux you'll be looking at mk/tools/tools.Linux.mk for how the native awk is found

To be continued.

@sskras
Copy link
Author

sskras commented Mar 25, 2024

Just to document my latest finding. It looks like the path for AWK is decided in mk/tools/replace.mk:

    226 .if !defined(TOOLS_IGNORE.awk) && !empty(_USE_TOOLS:Mawk)
    227 .  if ${PKGPATH} == lang/nawk
    228 MAKEFLAGS+=                     TOOLS_IGNORE.awk=
    229 .  elif !empty(_TOOLS_USE_PKGSRC.awk:M[yY][eE][sS])
    230 TOOLS_DEPENDS.awk?=             nawk>=20040207:../../lang/nawk
    231 TOOLS_CREATE+=                  awk
    232 TOOLS_PATH.awk=                 ${TOOLBASE}/bin/nawk
    233 .  endif
    234 .endif

... with TOOLBASE being set in mk/bsd.prefs.mk:

    607 LOCALBASE?=             /usr/pkg
    608 TOOLBASE:=              ${LOCALBASE}

netbsd-srcmastr pushed a commit that referenced this issue Mar 25, 2024
Changelog:

Version 4.3.1+dbg-1.6 (2022-01-22)

    Allow --profile to take an argument to specify callgrind for json PR #132 (Jayson Messenger)
    Fix bug in info var PR #113
    Typos in make.1 Issue #110
    Support gcc-10 Issue #109
    Document debugger commands
    Debugger commands are in separate .c files (not inlined, .h included)

I'd like to thank Marten Cassel for his sponsorship.

Version 4.3.1+dbg-1.5 (2020-03-15) Ron Frankel

Rebase code on GNU Make 4.3.

This was made possible by Thomas Kupper (boretom) who did all the heavy lifting while rocky looked on in amazement.

Thomas also added OS/X CI testing, and tested on the BSD-ish platforms.

With this release, we've started redoing the texinfo documentation in sphinx and put on readthedocs. Since sphinx supports output into LaTeX, TeXinfo, pdf, epub, and info as well as HTML, the TeXinfo document will be removed when the sphinx documentation is more stable. I expect the in next release to work on this.

Similarly, you'll see that README.md has been redone and includes screencasts. The install docs have also been corrected and expanded.
New and Changed Features

    --search parent -- I really like this one.
    --tasks -- the definition of a "tasks" has been simplified. Now, it is simply a target that has a description comment (#:) before it. After many years of using this myself, I highly encourage people to start using description comments more. Output from running remake --tasks is nicer because we use spaces to align columns rather than tabs.

Debugger Changes

    Commands with file expansion now use glob(), not word_expand() (Thomas did this too). Previously source used to POSIX.1-2008 wordexp(), but this is not available on BSD-ish systems. glob() is more general, and GNU make ships with its own glob() function when none is provided in the underlying OS library.
    load command added eval command removed. eval never worked and it attempted to be the same thing as load; load is the gdb name.
    $(debugger) function fixed. This function gave an virtual memory exhausted on exit. This has been fixed. The required parameter for this function, a tag name, is now shown on entry
    info tasks has been added. It is basically the same thing as remake --tasks
    Description lines are now shown on in info targets and list commands
    The debugger now disallows any "running" command inside post-mortem debugging
    Help as shown inside the debugger has been greatly expanded and more closely matches the sphinx docs. We now show in help text the short command name and any aliases attached to the command
@dhgutteridge
Copy link
Member

It seems mk/tools/tools.Linux.mk tries to find "gawk" specifically, and otherwise will not assign anything. That whole file basically assumes GNU tooling. (You kind of already partly answered your own question there in #130 (comment) .)

A dependency may indeed be specified in mk/tools/replace.mk where you quote, in certain contexts only.

(If you're looking for a single place where everything is revealed, well, a lot of the pkgsrc infrastructure isn't like that. It's more complicated.)

ricardobranco777 pushed a commit to ricardobranco777/pkgsrc that referenced this issue Mar 31, 2024
Also update bundled PEAR package Archive_Tar to 1.5.0.

PEAR 1.10.15 (2024-03-09)

* PR NetBSD#132: cleanup uneeded test
* PR NetBSD#135: Fix PHP Deprecated: Calling get_class() without arguments

Archive_Tar 1.5.0 (2024-03-16)

* PHP Version: PHP 5.2.0 or newer
* PEAR Package: PEAR Installer 1.9.0 or newer
@sskras sskras changed the title awk detection is broken on a Busybox- and Musl-based distro awk detection is broken on a Busybox- and Musl-based distros Apr 20, 2024
@sskras
Copy link
Author

sskras commented Apr 20, 2024

Confirmed on the following distros, attaching the logs inline:

Quoting @jperkin's IRC response and inviting for assistance:

firstly, does it not exist, or is /usr/pkg/bin/nawk there but somehow built with an invalid interpreter or something?

Surely the file does not exist yet:

# ls /usr/pkg/
ls: /usr/pkg/: No such file or directory

nawk is only bootstrapped if a native awk isn't found or is unsuitable, so I'd also check to see why your native awk isn't sufficient and if it can be provided/fixed instead

My impression is that nawk was not bootstrapped. That's the root problem I see.

then if it is required and is missing, I'd be looking at the bootstrap log to figure out why it failed and how that wasn't reported as a failure to the bootstrap script

Jonathan, I still doesn't understand that part. Sounds too abstract, maybe even ambiguous to me.
A guidance is needed :)

as this is Linux you'll be looking at mk/tools/tools.Linux.mk for how the native awk is found

The only awk references I found in mk/tools/tools.Linux.mk are:

    105 .  if exists(${_path}/gawk)
    106 TOOLS_PLATFORM.gawk?=           ${_path}/gawk
    107 .  endif
    ...
    280
    281 # assuming grep/sed/tar/awk are usually the GNU versions, is this safe?
    282
    283 .if !empty(TOOLS_PLATFORM.bison)
    284 TOOLS_PLATFORM.bison-yacc?=     ${TOOLS_PLATFORM.bison} -y
    285 .endif
    286
    287 TOOLS_PLATFORM.awk?=            ${TOOLS_PLATFORM.gawk}
    288

Which @dhgutteridge explained above:

It seems mk/tools/tools.Linux.mk tries to find "gawk" specifically, and otherwise will not assign anything. That whole file basically assumes GNU tooling.

@sskras
Copy link
Author

sskras commented Apr 20, 2024

Debian 12 also provides non-GNU implementation by default, the mawk. Interestingly, pkgsrc decides to bootstrap its' own nawk here just fine resulting in /usr/pkg/bin/nawk.

From 05-bootstrap-builds-nawk-on-Debian-12.log:

--- /root/debug/src/github.com/NetBSD/pkgsrc.git/debug/logs/03-bootstrap-misses-nawk-on-Alpine-Linux.log
+++ /root/debug/src/github.com/NetBSD/pkgsrc.git/debug/logs/05-bootstrap-builds-nawk-on-Debian-12.log
@@ -1,16 +1,16 @@
 ===> bootstrap command: bootstrap/bootstrap
-===> bootstrap started: Fri Apr 19 20:26:12 EEST 2024
+===> bootstrap started: Sat Apr 20 08:29:37 AM EEST 2024
 Working directory is: /root/debug/src/github.com/NetBSD/pkgsrc.git/work
  ...
-===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bmake && /bin/sh make-bootstrap.sh)
  ...
-===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bmake/bmake /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/bmake
+===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bmake && /usr/bin/sh make-bootstrap.sh)
  ...
+===> running: /usr/bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bmake/bmake /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/bmake
 ===> Building libnbcompat
-===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -d -o root -g root /root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat
-===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat; /bin/sh ./configure  -C --prefix=/usr/pkg --infodir=/usr/pkg/info --mandir=/usr/pkg/man --sysconfdir=/usr/pkg/etc --enable-bsd-getopt --enable-db && /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/bmake  -j1)
+===> running: /usr/bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -d -o root -g root /root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat
+===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat; /usr/bin/sh ./configure  -C --prefix=/usr/pkg --infodir=/usr/pkg/info --mandir=/usr/pkg/man --sysconfdir=/usr/pkg/etc --enable-bsd-getopt --enable-db && /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/bmake  -j1)
 configure: creating cache config.cache
-checking build system type... x86_64-pc-linux-musl
-checking host system type... x86_64-pc-linux-musl
+checking build system type... x86_64-pc-linux-gnu
+checking host system type... x86_64-pc-linux-gnu
 checking for gcc... gcc
 checking whether the C compiler works... yes
 checking for C compiler default output file name... a.out
@@ -265,13 +267,11 @@
 checking whether ln -s works... yes
 checking for ranlib... ranlib
 checking for gawk... no
-checking for mawk... no
-checking for nawk... no
-checking for awk... awk
+checking for mawk... mawk
 checking for ar... ar
 checking for fparseln in -lutil... no
-checking for grep that handles long lines and -e... /bin/grep
-checking for egrep... /bin/grep -E
+checking for grep that handles long lines and -e... /usr/bin/grep
+checking for egrep... /usr/bin/grep -E
 checking for _Bool... yes
 checking for stdbool.h that conforms to C99... yes
 checking for alloca.h... yes
  ...
 ranlib libnbcompat.a
+===> Bootstrapping awk
+===> running: /usr/bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -d -o root -g root /root/debug/src/github.com/NetBSD/pkgsrc.git/work/awk
+===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/work/awk && /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/bmake  -j1 -f Makefile CC="cc" CFLAGS="")
+cc   -c awkgram.tab.c
+cc   -c b.c
+cc   -c main.c
+cc   -c parse.c
+cc   -c proctab.c
+cc   -c tran.c
+cc   -c lib.c
+cc   -c run.c
+cc   -c lex.c
+cc  awkgram.tab.o b.o main.o parse.o proctab.o tran.o lib.o run.o lex.o   -lm -o a.out
+===> running: /usr/bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/awk/a.out /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/nawk
 ===> Bootstrapping pkgtools

I wonder what makes the bootstrap fail to reach (or just skip) this step on the topic distros.

@sskras
Copy link
Author

sskras commented Apr 20, 2024

OK, found the answer in bootstrap/bootstrap:

 737         # Debian/Ubuntu's awk is mawk, and mawk does not understand
 738         # some regexps used in pkgsrc/mk.
 739         if [ -f /etc/debian_version ]; then
 740                 need_awk=yes

So it's tied to the hardcoded OS name. Waiting for the suggestions on how to proceed in a clean way :)

@sskras sskras changed the title awk detection is broken on a Busybox- and Musl-based distros awk detection is broken on a Busybox- and Musl-based Linux distros Apr 24, 2024
netbsd-srcmastr pushed a commit that referenced this issue Apr 29, 2024
Changes in 2.2.5

    #117: Document that empty for Concurrently waits forever
    #120: Add ConcurrentlyE.
    #123: Fix failing concurrentlyE tests in older GHCs.
    #124: Allow hashable 1.4
    #126: Semigroup and Monoid instances for ConcurrentlyE
    #120: Add ConcurrentlyE
    #138: expose internals as Control.Concurrent.Async.Internal
    #131: Fix typos in docs
    #132: waitAny(Catch): clarify non-empty input list requirement
    #142: Add cancelMany
    #135, #145, #150: Support for GHC 9.4, 9.6, 9.8
    Document that empty for Concurrently waits forever.
netbsd-srcmastr pushed a commit that referenced this issue Apr 29, 2024
0.2.5.4
* Fix autolink parsing regression (#151). This affects autolinks with
  doubled internal line-ending punctuation characters.

0.2.5.3
* Fix rebase_relative_paths extension so it works with URLs with non-ASCII
  characters (#148). Previously these would not be properly detected as
  absolute URIs.

0.2.5.2
* Improve autolinks extension (#147). The autolinks extension was
  interacting badly with explicit links, To fix this we had to make
  autolink parsing a bit stricter than the GFM spec does. They allow
  unbalanced ) except at the end of a URL (which is defined as: followed by
  optional final punctuation then whitespace or eof). With this change, we
  don't allow unbalanced ) at all in raw URLs. This should not be a big
  problem in practice.
* Protect against quadratic generated table size explosion (Michael
  Howell). This commit adds a limit to the number of auto-completed cells
  around 200,000. The result is, in these original samples:

0.2.5.1
* Add test/alerts.md to extra-source-files in cabal file.

0.2.5
* Add support for alerts extension, supporting GitHub-style alerts
  (#132). https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax#alerts
  New module Commonmark.Extensions.Alerts [API change].
* Do not accept footnote labels with line breaks (Michael Howell).
* Parse [^ ] and [^] as links (Michael Howell). This is consistent with
  most other CommonMark parsers, even when they have support for footnotes
  turned on.
netbsd-srcmastr pushed a commit that referenced this issue May 2, 2024
0.2.2.1
    Remove nested footnotes (#138).

0.2.2
    Add support for alerts extension (#132).
netbsd-srcmastr pushed a commit that referenced this issue May 6, 2024
2024-05-05 -- 0.9.8

>>>>>>>>>>>>> SECURITY >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
  * Fixed: [CVE-2024-34402]
      Protect against integer overflow in ComposeQueryEngine
      (GitHub #183, GitHub #185)
  * Fixed: [CVE-2024-34403]
      Protect against integer overflow in ComposeQueryMallocExMm
      (GitHub #183, GitHub #186)
>>>>>>>>>>>>> SECURITY >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
  * Changed: Require CMake >=3.5.0 (GitHub #172)
  * Added: CMake option URIPARSER_SHARED_LIBS=(ON|OFF) to control,
      whether to produce a shared or static library for uriparser
      and that alone, falls back to standard BUILD_SHARED_LIBS
      if available, else defaults to "ON" (GitHub #169, GitHub #170)
  * Improved: Document that scheme-based normalization a la
      section 6.2.3 of RFC 3986 is a responsibility of the application
      using uriparser (GitHub #173, GitHub #174)
  * Improved: Document supported code points for functions uriEscape(Ex)W
      (GitHub #171, GitHub #175)
  * Infrastructure: Update Clang from 15 to 18 (GitHub #161, GitHub #187)
  * Infrastructure: Adapt to breaking changes in Clang packaging (GitHub #160)
  * Infrastructure: Get sanitizer CFLAGS and LDFLAGS back in sync (GitHub #161)
  * Infrastructure: Pin GitHub Actions to specific commits for security
      (GitHub #165)
  * Soname: 1:31:0 — see https://verbump.de/ for what these numbers do

2022-10-05 -- 0.9.7

  * Fixed: Multiple issues with IPv6 and IPvFuture literal parsing
      (GitHub #146, GitHub #150)
      Thanks to Scallop Ye for the report and the pull request!
  * Fixed: Fix symbol visibility for -DBUILD_SHARED_LIBS=OFF (GitHub #139,
      GitHub #141); thanks to Mariusz Zaborski for the report!
  * Fixed: For MinGW, use size_t for inet_ntop declaration and fix macro
      checks for both MinGW and mingw-w64 (GitHub #131)
  * Fixed: Compiler warnings (GitHub #132, GitHub #152)
  * Improved: Use name UriConfig.h rather than generic config.h for the
      config header file to avoid name clashes and also include it through
      "UriConfig.h" with quotes rather than <UriConfig.h> so that it is found
      in quote path locations (GitHub #149)
      Thanks to Gaspard Petit for bringing this up!
  * Improved: Document need for UriConfig.h in UriMemory.c (GitHub #136)
  * Infrastructure: Add (support for) Visual Studio 17/2022 (GitHub #152)
  * Infrastructure: Drop (support for) Visual Studio <=14/2015 (GitHub #152)
  * Infrastructure: Update Clang from 13 to 15 (GitHub #143, GitHub #151)
  * Infrastructure: Make MinGW with 32bit Wine on Ubuntu 20.04 possible
      (GitHub #142, GitHub #144, GitHub #145)
  * Soname: 1:30:0 — see https://verbump.de/ for what these numbers do
@alarixnia
Copy link
Member

So it's tied to the hardcoded OS name. Waiting for the suggestions on how to proceed in a clean way :)

The main thing we would avoid changing (active users and all...) is Enterprise Linux. Perhaps you can tell us if GNU awk is specifically identifiable on distributions like CentOS @sskras (perhaps installed as "gawk" in some directory)? Otherwise we can set need_awk=yes on all Linux...

@sskras
Copy link
Author

sskras commented Jul 6, 2024

Thanks @alarixnia. I think I would rather:

  1. collect the needed AWK features,
  2. detect during bootstrap if any on them is missing,
  3. only then fall back to need_awk=yes.

Perhaps you can tell us if GNU awk is specifically identifiable on distributions like CentOS @sskras (perhaps installed as "gawk" in some directory)?

On CentOS 9 it's this:

saukrs@mp-client:~$ type awk gawk
awk is /usr/bin/awk
gawk is /usr/bin/gawk

saukrs@mp-client:~$ ll /usr/bin/gawk /usr/bin/awk
lrwxrwxrwx. 1 root root      4 Feb 16  2022 /usr/bin/awk -> gawk*
-rwxr-xr-x. 1 root root 714976 Feb 16  2022 /usr/bin/gawk*

saukrs@mp-client:~$ . /etc/os-release; echo $PRETTY_NAME
CentOS Stream 9

Not sure if that helps.

@sskras
Copy link
Author

sskras commented Jul 6, 2024

The same issue happens on Adelie Linux 1.0-beta5, also Musl-based distro (but no Busybox here):
10-bootstrap-misses-nawk-on-Adelie-Linux.log

@sskras sskras changed the title awk detection is broken on a Busybox- and Musl-based Linux distros awk detection is broken on Musl-based Linux distros Jul 6, 2024
netbsd-srcmastr pushed a commit that referenced this issue Oct 21, 2024
1.8.2 (2024-09-24)

What's Changed

* Drop commented-out line by @olleolleolle in #108
* Add Ruby 3.1 & 3.2 to CI matrix by @tricknotes in #109
* Fix/redos by @ooooooo-q in #114
* Raise HTTPStatus::BadRequest for requests with invalid/duplicate
  content-length headers by @jeremyevans in #120
* Bump actions/checkout from 3 to 4 by @dependabot in #121
* Improve CI by @hsbt in #123
* Fix WEBrick::TestFileHandler#test_short_filename test not working on mswin
  by @KJTsanaktsidis in #128
* Fix bug chunk extension detection by @jeremyevans in #125
* Fix CI. by @ioquatix in #131
* Merge multiple cookie headers, preserving semantic correctness. by
  @ioquatix in #130
* Test on macos-latest by @byroot in #132
* Require CRLF line endings in request line and headers by @jeremyevans in
  #138
* Prefer squigly heredocs. by @ioquatix in #143
* Only strip space and horizontal tab in headers by @jeremyevans in #141
* Treat missing CRLF separator after headers as an EOFError by @jeremyevans
  in #142
* Return 400 response for chunked requests with unexpected data after chunk
  by @jeremyevans in #136
* Fix reference to URI::REGEXP::PATTERN::HOST by @casperisfine in #144
* Prevent request smuggling by @jeremyevans in #146

New Contributors

* @tricknotes made their first contribution in #109
* @ooooooo-q made their first contribution in #114
* @KJTsanaktsidis made their first contribution in #128
* @byroot made their first contribution in #132
* @casperisfine made their first contribution in #144
netbsd-srcmastr pushed a commit that referenced this issue Oct 26, 2024
(inst/NEWS)
Version 0.7-7
    o Replace Calloc and Free with R_Calloc and R_Free, respectively,
    to conform STRICT.

Version 0.7-6
    o   Fix bugs found by warnings and suppress other nonsesical warnings
    o   Fix partial match warning #132
    o   exclude dropped column from column listing #130
netbsd-srcmastr pushed a commit that referenced this issue Nov 11, 2024
2.2 (2024-08-27)

* WebUI: More correct filtering reports by disposition value
* WebUI: add actual fail count to the report list (#96)
* WebUI: implemented use of abbreviated form for numbers in the list of
  reports
* WebUI: added a toolbar and column management to the report list. The
  default column set was changed
* Implemented destroying user sessions if user_management is set to false
* Added ignoring unknown tags in XML report files
* Fixed undefined property usage in utils/reports_cleaner.php (#131)
* Fixed an error occuring when adding a user without the level parameter via
  utils/users_admin.php
* Improved error message when passing invalid time to Report dialog
* Updated the "Utilities" section in README.md
* Other minor changes and updates
* Some code optimization and refactoring

2.2.1 (2024-08-28)

* Fixed an error "Undefined array key" occurring when processing an
  incorrect report file (#132)
netbsd-srcmastr pushed a commit that referenced this issue Nov 27, 2024
Changes in [1.11.86](https://github.com/element-hq/element-web/releases/tag/v1.11.86) (2024-11-19)
==================================================================================================
## ✨ Features

* Deduplicate icons using Compound Design Tokens ([#28419](element-hq/element-web#28419)). Contributed by @t3chguy.
* Let widget driver send error details ([#28357](element-hq/element-web#28357)). Contributed by @AndrewFerr.
* Deduplicate icons using Compound Design Tokens ([#28381](element-hq/element-web#28381)). Contributed by @t3chguy.
* Auto approvoce `io.element.call.reaction` capability for element call widgets ([#28401](element-hq/element-web#28401)). Contributed by @toger5.
* Show message type prefix in thread root \& reply previews ([#28361](element-hq/element-web#28361)). Contributed by @t3chguy.
* Support sending encrypted to device messages from widgets ([#28315](element-hq/element-web#28315)). Contributed by @hughns.

## 🐛 Bug Fixes

* Feed events to widgets as they are decrypted (even if out of order) ([#28376](element-hq/element-web#28376)). Contributed by @robintown.
* Handle authenticated media when downloading from ImageView ([#28379](element-hq/element-web#28379)). Contributed by @t3chguy.
* Ignore `m.3pid_changes` for Identity service 3PID changes ([#28375](element-hq/element-web#28375)). Contributed by @t3chguy.
* Fix markdown escaping wrongly passing html through ([#28363](element-hq/element-web#28363)). Contributed by @t3chguy.
* Remove "Upgrade your encryption" flow in `CreateSecretStorageDialog` ([#28290](element-hq/element-web#28290)). Contributed by @florianduros.


Changes in [1.11.85](https://github.com/element-hq/element-web/releases/tag/v1.11.85) (2024-11-12)
==================================================================================================
# Security
- Fixes for [CVE-2024-51750](https://www.cve.org/CVERecord?id=CVE-2024-51750) / [GHSA-w36j-v56h-q9pc](GHSA-w36j-v56h-q9pc)
- Fixes for [CVE-2024-51749](https://www.cve.org/CVERecord?id=CVE-2024-51749) / [GHSA-5486-384g-mcx2](GHSA-5486-384g-mcx2)
- Update JS SDK with the fixes for [CVE-2024-50336](https://www.cve.org/CVERecord?id=CVE-2024-50336) / [GHSA-xvg8-m4x3-w6xr](GHSA-xvg8-m4x3-w6xr)


Changes in [1.11.84](https://github.com/element-hq/element-web/releases/tag/v1.11.84) (2024-11-05)
==================================================================================================
## ✨ Features

* Remove abandoned MSC3886, MSC3903, MSC3906 implementations ([#28274](element-hq/element-web#28274)). Contributed by @t3chguy.
* Update to React 18 ([#24763](element-hq/element-web#24763)). Contributed by @t3chguy.
* Deduplicate icons using Compound ([#28239](element-hq/element-web#28239)). Contributed by @t3chguy.
* Replace legacy Tooltips with Compound tooltips ([#28231](element-hq/element-web#28231)). Contributed by @t3chguy.
* Deduplicate icons using Compound Design Tokens ([#28219](element-hq/element-web#28219)). Contributed by @t3chguy.
* Add reactions to html export ([#28210](element-hq/element-web#28210)). Contributed by @langleyd.
* Remove feature\_dehydration ([#28173](element-hq/element-web#28173)). Contributed by @florianduros.

## 🐛 Bug Fixes

* Remove upgrade encryption in `DeviceListener` and `SetupEncryptionToast` ([#28299](element-hq/element-web#28299)). Contributed by @florianduros.
* Fix 'remove alias' button in room settings ([#28269](element-hq/element-web#28269)). Contributed by @Dev-Gurjar.
* Add back unencrypted path in `StopGapWidgetDriver.sendToDevice` ([#28295](element-hq/element-web#28295)). Contributed by @florianduros.
* Fix other devices not being decorated as such ([#28279](element-hq/element-web#28279)). Contributed by @t3chguy.
* Fix pill contrast in invitation dialog ([#28250](element-hq/element-web#28250)). Contributed by @florianduros.
* Close right panel chat when minimising maximised voip widget ([#28241](element-hq/element-web#28241)). Contributed by @t3chguy.
* Fix develop changelog parsing ([#28232](element-hq/element-web#28232)). Contributed by @t3chguy.
* Fix Ctrl+F shortcut not working with minimised room summary card ([#28223](element-hq/element-web#28223)). Contributed by @t3chguy.
* Fix network dropdown missing checkbox \& aria-checked ([#28220](element-hq/element-web#28220)). Contributed by @t3chguy.


Changes in [1.11.83](https://github.com/element-hq/element-web/releases/tag/v1.11.83) (2024-10-29)
==================================================================================================
## ✨ Features

* Enable Element Call by default on release instances ([#28314](element-hq/element-web#28314)). Contributed by @t3chguy.



Changes in [1.11.82](https://github.com/element-hq/element-web/releases/tag/v1.11.82) (2024-10-22)
==================================================================================================
## ✨ Features

* Deduplicate more icons using Compound Design Tokens ([#132](element-hq/matrix-react-sdk#132)). Contributed by @t3chguy.
* Always show link new device flow even if unsupported ([#147](element-hq/matrix-react-sdk#147)). Contributed by @t3chguy.
* Update design of files list in right panel ([#144](element-hq/matrix-react-sdk#144)). Contributed by @t3chguy.
* Remove feature\_dehydration ([#138](element-hq/matrix-react-sdk#138)). Contributed by @florianduros.
* Upgrade emojibase-bindings and remove local handling of emoticon variations ([#127](element-hq/matrix-react-sdk#127)). Contributed by @langleyd.
* Add support for rendering media captions ([#43](element-hq/matrix-react-sdk#43)). Contributed by @tulir.
* Replace composer icons with Compound variants ([#123](element-hq/matrix-react-sdk#123)). Contributed by @t3chguy.
* Tweak default right panel size to be 320px except for maximised widgets at 420px ([#110](element-hq/matrix-react-sdk#110)). Contributed by @t3chguy.
* Add a pinned message badge under a pinned message ([#118](element-hq/matrix-react-sdk#118)). Contributed by @florianduros.
* Ditch right panel tabs and re-add close button ([#99](element-hq/matrix-react-sdk#99)). Contributed by @t3chguy.
* Force verification even for refreshed clients ([#44](element-hq/matrix-react-sdk#44)). Contributed by @dbkr.
* Update emoji text, border and background colour in timeline ([#119](element-hq/matrix-react-sdk#119)). Contributed by @florianduros.
* Disable ICE fallback based on well-known configuration ([#111](element-hq/matrix-react-sdk#111)). Contributed by @t3chguy.
* Remove legacy room header and promote beta room header ([#105](element-hq/matrix-react-sdk#105)). Contributed by @t3chguy.
* Respect `io.element.jitsi` `useFor1To1Calls` in well-known ([#112](element-hq/matrix-react-sdk#112)). Contributed by @t3chguy.
* Use Compound close icon in favour of mishmash of x/close icons ([#108](element-hq/matrix-react-sdk#108)). Contributed by @t3chguy.

## 🐛 Bug Fixes

* Correct typo in option documentation ([#28148](element-hq/element-web#28148)). Contributed by @AndrewKvalheim.
* Revert #124 and #135 ([#139](element-hq/matrix-react-sdk#139)). Contributed by @dbkr.
* Add aria-label to e2e icon ([#136](element-hq/matrix-react-sdk#136)). Contributed by @florianduros.
* Fix bell icons on room list hover being black squares ([#135](element-hq/matrix-react-sdk#135)). Contributed by @dbkr.
* Fix vertical overflow on the mobile register screen ([#137](element-hq/matrix-react-sdk#137)). Contributed by @langleyd.
* Allow to unpin redacted event ([#98](element-hq/matrix-react-sdk#98)). Contributed by @florianduros.
netbsd-srcmastr pushed a commit that referenced this issue Nov 27, 2024
v0.8.2

Mostly frontend improvements, the highlights being improved video/audio sync and more flexible

input/hotkey mapping. Also adds Genesis overclocking support.

Input/hotkey configuration is not compatible with previous versions; input configuration is now stored in a different format, and all input-related settings will one-time revert to defaults if using a config file from a previous version

Save states are not compatible with previous versions
New Features

    Video/audio sync improvements which should enable significantly improved frame pacing without needing to rely on 60Hz VSync (which can cause very noticeable input latency on some platforms)
        Added a new "frame time sync" option that uses the host system clock to match the emulated system's framerate and frame timing as closely as possible without relying on host GPU synchronization (i.e. VSync)
        Added a new option for dynamic audio resampling ratio, which periodically adjusts the audio resampling ratio to try and avoid audio buffer underflows and overflows (which both cause audio popping)
            This is implemented in a very conservative way in order to avoid audible differences in audio pitch, so it is not completely guaranteed to prevent audio buffer underflow/overflow
        Audio sync now checks the audio buffer size every 16 samples enqueued rather than only checking once per frame, which should significantly reduce stuttering when audio sync is enabled without VSync or frame time sync
        Adjusted default sync/audio settings values to hopefully make stuttering and audio popping less likely when running with default settings
        In the GUI, video/audio sync settings have been moved to a new window under Settings > Synchronization
    Input mapping overhaul to make input mapping/configuration more flexible (#134 / #137)
        Keyboard and gamepad settings are no longer separate configurations; each system now supports up to 2 mappings for each emulated button where each mapping can be a keyboard key, a gamepad input, or a mouse button
        Key/input/button combinations (2 or 3 simultaneous inputs) are now supported for mappings in addition to individual keys/inputs
        Hotkeys can now be mapped to gamepad inputs, mouse buttons, and combinations in addition to individual keyboard keys
        Each input settings window now has a button to apply one of two keyboard presets for P1 inputs, one with arrow keys mapped to the d-pad and one with WASD mapped to the d-pad
    Added a new set of hotkeys for saving/loading specific save state slots (#134)
    (Genesis / Sega CD / 32X) Added an option to overclock the main Genesis CPU (the 68000) by decreasing the master clock divider, which can reduce or eliminate slowdown in games (#133)
        Note that this is a fairly naive form of overclocking that works very well in many games but very poorly in some games; use with caution
        As far as SCD/32X, from my testing overclocking the 68000 almost always causes problems in 32X games (which are normally bottlenecked on SH-2 speed anyway), but it does fix slowdown in some Sega CD games
    (SMS / Game Gear) Replaced the "double Z80 CPU speed" setting with an option to overclock at finer granularity by decreasing the Z80 master clock divider
        Same caveat as above regarding this form of overclocking working very poorly in some games, and this is more of an issue for SMS/Game Gear than it is for Genesis/Sega CD
    Added an option to only hide the mouse cursor when in fullscreen, in addition to the previous settings of "always hide" and "never hide"
    Added an option to change the fullscreen mode from borderless to exclusive
    Added an option to change the audio output frequency from 48000 Hz to 44100 Hz

Improvements

    (Genesis / Sega CD) Slightly improved performance by advancing the emulated clock in larger intervals while a long VDP DMA is in progress
    (32X) Slightly improved performance by optimizing SH-2 instruction decoding
    (GB) Improved video frame delivery behavior when the PPU is powered off to make it play a little nicer with VSync and frame time sync
    The emulator window is now explicitly focused/raised when a game is loaded; previously this wouldn't always happen automatically, particularly on Windows

Fixes

    (Sega CD) Slightly extended the delay between a game sending a CDD Play/Read command and the CD drive reading the first sector; this fixes Time Gal having excruciatingly long "load times"
        "Load times" in quotes because the game was actually getting confused and repeatedly re-reading the same CD-ROM sectors until various interrupts happened to trigger at exactly the right times relative to each other
    (Sega CD) Fixed a bug where some backend settings would not correctly persist after loading a save state (they would temporarily revert to what they were when the save state was created)

v0.8.1

Small release with mostly bugfixes and a few save state-related features

SMS/Game Gear and NES save states are not compatible with previous versions, other systems' save states should be compatible
Features

    Made the game save file and save state locations configurable (#132)
        Can be set to the same folder as the ROM image (same as previous behavior), subdirectories in the emulator folder, or custom paths
    Added a new --load-save-state <SLOT> command-line arg to load a specific save state slot at game launch (#132)
    Added an option to attempt to load the most recently saved state when launching a game
        If this option is set when there are no save states or the most recent state cannot be loaded, the game will boot normally
    (Game Gear) Added an option to render at SMS resolution (256x192) instead of native resolution (160x144)
        The expanded parts of the frame often contain garbage because they weren't meant to be visible, but they sometimes contain an expanded playfield

Fixes

    (Sega CD) Fixed a bug where loading a save state could possibly crash the emulator due to a stack overflow; this was particularly likely to happen on Windows due to the small default stack size
        This was caused by the state deserialization code inadvertently deserializing some very large arrays into the stack before moving them to the heap, rather than deserializing directly into the heap
    (SMS/Game Gear) Fixed the VDP display disabled implementation so that it properly blanks the display rather than leaving the previous frame onscreen
    (SMS) Fixed the NTSC/PAL and SMS Model settings not having any effect when loading a game from a .zip/.7z file rather than a .sms file
    (NES) Fixed multiple bugs related to how the PPU determines what color to display when rendering is disabled while PPUADDR points to palette RAM; this fixes Micro Machines having a solid gray bar in the middle of the title screen, as well as several test ROMs that rely on this hardware quirk for high-color display (#53 / #55 / #56)
    GUI: Saving or loading a save state slot from the GUI window now also changes the selected save state slot
    CLI: Fixed

    the 32X option missing from the help text for the --hardware arg (#131)
    The video memory viewer window now renders without VSync; this fixes likely stuttering and audio popping while the memory viewer window is open
netbsd-srcmastr pushed a commit that referenced this issue Dec 2, 2024
# wk 0.9.4

- Ensure package tests pass against sf 1.0-18 (#224, #225).

# wk 0.9.3

- Ensure package compiles with `STRICT_R_HEADERS=1` (#222).

# wk 0.9.2

- Add `wk_crs()` and `wk_set_crs()` methods for `bbox` (#213)
- Fix wk_trans inconsistent meta flags handling (#217)
- Ensure package builds on arm64 for Windows (#220)

# wk 0.9.1

- Fix format strings/arguments for R-devel (#209).

# wk 0.9.0

## Breaking changes

- The common well-known binary representation of POINT EMPTY (i.e.,
  POINT (nan nan)) is now handled as POINT EMPTY allowing empty points
  to roundtrip through `wkb()` vectors (#196, #204).
- `xy(NA, NA)` is now read as a null feature instead of POINT EMPTY. This
  preserves the invariant that null features can also be identified using
  `is.na()` (#205).
- `xy(NaN, NaN)` is now read as POINT EMPTY and `is.na(xy(NaN, NaN))`
  now returns `FALSE`. This means that both EMPTY and null points can roundtrip
  through `xy()` (#205).

## Bugfixes and improvements

- `wk_meta()` now contains a new column `is_empty`, which is `TRUE`
  for any feature that contains at least one non-empty coordinate. This allows
  more efficient detection of features with zero coordinates (#197, #199).
- Updated PROJ data to use the latest pull of the database packaged with
  PROJ 9.3.0 (#201).
- The wk package now compiles once again on gcc 4.8 (#203, #206).
- Fixed `sfc_writer()` to correctly attach the `classes` attribute to
  sfc output with mixed geometry types (#195).
- Function `sfc_writer()` now has an argument `promote_multi` to write any
  input as the MULTI variant. This makes it more likely that an input vector
  will be read as a single geometry type (#198).
- The `wk_collection_filter()` now correctly increments the `part_id` when
  calling the child handler (@brownag, #194).

# wk 0.8.0

* Added `wkb_to_hex()` (@anthonynorth, #183).
* Implemented `vctrs::vec_proxy_equal()` for `wkb()` vctrs
  (@anthonynorth, #183).
* Fixed `sfc_writer()`, which had returned NULL for some inputs
  (e.g., via `wk_collection()`) (@anthonynorth, #182, #186).
* Added `wk_clockwise()` and `wk_counterclockwise()` to re-wind polygon rings
  (@anthonynorth, #188).
* New replacement-function mode for `wk_coords<-()` for in-place modification
  of coordinates (@mdsumner, #187).
* New function `wk_trans_explicit()` migrated from crs2crs (@mdsumner, #187).

# wk 0.7.3

* Fix tests for updated waldo package (#178).

# wk 0.7.2

* Fix use-after-free warnings.

# wk 0.7.1

* Fix implicit reliance on error `as.data.frame.default()`,
  which no longer occurs in r-devel (#166).

# wk 0.7.0

* Remove legacy headers that are no longer used by any downstream package
  (#146).
* `validate_wk_wkt()` now errors for an object that does not inherit from
  'wk_wkt' (#123, #146).
* Added `wk_crs_projjson()` to get a JSON representation of a CRS object.
  To make lookup possible based on shortcut-style CRS objects (e.g.,
  `"EPSG:4326"` or `4326`), added data objects `wk_proj_crs_view` and
  `wk_proj_crs_json` that contain cached versions of rendered PROJJSON
  based on the latest PROJ version (#147).
* Added a `wk_crs_proj_definition()` method for `wk_crs_inherit()` (#136,
  #147).
* Conversion to sf now uses the `sfc_writer()` for all wk classes, making
  conversions faster and fixing at least one issue with conversion of NA
  geometries to sf (#135).
* `wk_plot()` now plots `NULL`/`NA` geometries and mixed geometry types
  more reliably (#142, #143, #149).
* Exported EMPTY geometries to well-known text now include dimension
  (e.g., `POINT Z EMPTY`) (#141, #150).
* Fixed bug where `wk_polygon()` doubled some points when the input contained
  closed rings (#134, #151).
* Fixed bug where `wk_count()` exposed uninitialized values for empty input
  (#139, #153).
* The `xy_writer()` now opportunistically avoids allocating vectors for Z
  or M values unless they are actually needed (#131, #154).
* Added example WKT for all geometry types and dimensions plus helper
  `wk_example()` to access them and set various properties (#155).
* Fixes warnings when compiling with `-Wstrict-prototypes` (#157, #158).
* Removed `wk_chunk_map_feature()` in favour of using chunking strategies
  directly (#132, #159).
* Optimized `wk_coords()` for `xy()` objects (#138, #160).
* Added accessor methods for record-style vectors: `rct_xmin()`, `rct_xmax()`,
  `rct_ymin()`, `rct_ymax()`, `rct_width()`, `rct_height()`, `crc_center()`,
  `crc_x()`, `crc_y()`, `crc_r()`, `xy_x()`, `xy_y()`, `xy_z()`, and `xy_m()`
  (#144, #161).
* Added rectangle operators `rct_intersects()`, `rct_contains()`,
  and `rct_intersection()` (#161).

# wk 0.6.0

* Fixed `wk_affine_rescale()` to apply the translate and scale
  operations in the correct order (#94).
* Add `wk_handle_slice()` and `wk_chunk_map_feature()` to support
  a chunk + apply workflow when working with large vectors (#101, #107).
* C and R code was rewritten to avoid materializing ALTREP vectors
  (#103, #109).
* Added a `wk_crs_proj_definition()` generic for foreign CRS objects
  (#110, #112).
* Added `wk_crs_longlat()` helper to help promote authority-compliant
  CRS choices (#112).
* Added `wk_is_geodesic()`, `wk_set_geodesic()`, and argument `geodesic`
  in `wkt()` and `wkb()` as a flag for objects whose edges must
  be interpolated along a spherical/ellipsoidal trajectory (#112).
* Added `sf::st_geometry()` and `sf::st_sfc()` methods for wk geometry
  vectors for better integration with sf (#113, #114).
* Refactored well-known text parser to be more reusable and faster
  (#115, #104).
* Minor performance enhancement for `is.na()` and `validate_wk_wkb()`
  when called on a very long `wkb()` vector (#117).
* Fixed issue with `validate_wk_wkb()` and `validate_wk_wkt()`, which failed
  for most valid objects (#119).
* Added `wk_envelope()` and `wk_envelope_handler()` to compute feature-wise
  bounding boxes (#120, #122).
* Fixed headers and tests to pass on big endian systems (#105, #122).
* Incorporated the geodesic attribute into vctrs methods, data frame
  columns, and bbox/envelope calculation (#124, #125).
* Fix `as_xy()` for nested data frames and geodesic objects (#126, #128).
* Remove deprecated `wkb_problems()`, `wkt_problems()`, `wkb_format()`,
  and `wkt_format()` (#129).
* `wk_plot()` is now an S3 generic (#130).

# wk 0.5.0

* Fixed bugs relating to the behaviour of wk classes as
  vectors (#64, #65, #67, #70).
* `crc()` objects are now correctly exported as polygons
  with a closed loop (#66, #70).
* Added `wk_vertices()` and `wk_coords()` to extract individual
  coordinate values from geometries with optional identifying
  information. For advanced users, the `wk_vertex_filter()`
  can be used as part of a pipeline to export coordinates
  as point geometries to another handler (#69, #71).
* Added `wk_flatten()` to extract geometries from collections.
  For advanced users, the `wk_flatten_filter()` can be used as
  part of a pipeline (#75, #78).
* `options("max.print")` is now respected by all vector classes
  (#72, #74).
* Moved implementation of plot methods from wkutils to wk to
  simplify the dependency structure of both packages (#80, #76).
* Added `wk_polygon()`, `wk_linestring()`, and `wk_collection()`
  to construct polygons, lines, and collections. For advanced
  users, `wk_polygon_filter()`, `wk_linestring_filter()`, and
  `wk_collection_filter()` can be used as part of a pipeline
  (#77, #84).
* Added a C-level transform struct that can be used to simplify
  the the common pattern of transforming coordinates. These
  structs can be created by other packages; however, the
  `wk_trans_affine()` and `wk_trans_set()` transforms are
  also built using this feature. These are run using the
  new `wk_transform()` function and power the new
  `wk_set_z()`, `wk_set_m()`, `wk_drop_z()`, `wk_drop_m()`,
  functions (#87, #88, #89).

# wk 0.4.1

* Fix LTO and MacOS 3.6.2 check errors (#61).

# wk 0.4.0

* Removed `wksxp()` in favour of improved `sf::st_sfc()` support
  (#21).
* Rewrite existing readers, writers, and handlers, using
  a new C API (#13).
* Use new C API in favour of header-only approach for all
  wk functions (#19, #22).
* Use cpp11 to manage safe use of callables that may longjmp
  from C++.
* Vector classes now propagate `attr(, "crs")`, and check
  that operations that involve more than one vector have
  compatable CRS objects as determined by `wk_crs_equal()`.
* Added an R-level framework for other packages to implement
  wk readers and handlers: `wk_handle()`, `wk_translate()`,
  and `wk_writer()` (#37).
* Added a native reader and writer for `sf::st_sfc()` objects
  and implemented R-level generics for sfc, sfg, sf, and bbox
  objects (#28, #29, #38, #45).
* Implement `crc()` vector class to represent circles (#40).
* Added a 2D cartesian bounding box handler (`wk_bbox()`) (#42).
* Refactored unit tests reflecting use of the new API and
  for improved test coverage (#44, #45, #46).
* Added `wk_meta()`, `wk_vector_meta()`, and `wk_count()` to
  inspect properties of vectors (#53).
* Modified all internal handlers such that they work with vectors
  of unknown length (#54).

# wk 0.3.4

* Fixed reference to `wkutils::plot.wk_wksxp()`, which
  no longer exists.

# wk 0.3.3

* Fixed WKB import of ZM geometries that do not use EWKB.
* Added `xy()`, `xyz()`, `xym()` and `xyzm()` classes
  to efficiently store point geometries.
* Added the `rct()` vector class to efficiently store
  two-dimensional rectangles.
* Fixed the CRAN check  failure caused by a circular
  dependency with  the wkutils package.
* Added S3 methods to coerce sf objects to and from
  `wkt()`, `wkb()` and `wksxp()`.

# wk 0.3.2

* Fixed EWKB output for collections and multi-geometries
  that included SRID (#3).
* Fixed CRAN check errors related to exception handling on
  MacOS/R 3.6.2.

# wk 0.3.1

* Added a `NEWS.md` file to track changes to the package.
@sskras
Copy link
Author

sskras commented Dec 15, 2024

Well, I doubt 3309ac4 and www/R-sass in general has anything to do with detection of AWK.
I think this issue was closed by mistake.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants