Regarding the inclusion of OpenBangla Keyboard into Debian package repository #207

mominul · 2021-04-19T19:58:31Z

mominul
Apr 19, 2021
Maintainer

Hi, @gunnarhj!
Thanks again for your warm response to my email! Let's discuss the issues regarding including OpenBangla Keyboard into the Debian package repository here! 😊

The 'license' under which the files in src/frontend/3rdParty are
distributed does not comply with the Debian Free Software Guidelines
(DFSG). It means that OpenBangla Keyboard would need to be placed in the
non-free section instead of Debian main. On the Ubuntu side it would be
placed in multiverse rather than universe.

That'd be bad. How about introducing a compile-time variable that will control the compilation and linking of the 3rdParty code for now? That code is used for update checking currently. I will remove this functionality when OpenBangla Keyboard gets included in the package repository and seamless update process is ensured.

there is a tools/LICENSE file with a MIT
license. Does that license apply to any of the source files?

No, it doesn't. Actually, that applies only to the scripts in that folder that are mainly used to create and deploy OpenBangla Keyboard packages on Github Actions.

Otherwise, considering that you have created .deb files already, it
would be good to see the source from such a build to see the build
dependencies for instance.

Currently, the build dependencies are build-essential rustc cargo cmake libibus-1.0-dev qt5-default libzstd-dev.
The deb and other packages are created through CMake which uses CPack to build the package with the following configurations:

OpenBangla-Keyboard/CMakeLists.txt

Lines 80 to 118 in 77255c7

	set(CPACK_PACKAGE_DESCRIPTION_SUMMARY "OpenSource Bengali input method")
	set(CPACK_PACKAGE_VENDOR "OpenBangla Team")
	set(CPACK_DEBIAN_PACKAGE_MAINTAINER "OpenBangla Team <[email protected]>")
	set(CPACK_PACKAGE_DESCRIPTION_FILE "${CMAKE_SOURCE_DIR}/data/Readme.txt")
	set(CPACK_RESOURCE_FILE_LICENSE "${CMAKE_SOURCE_DIR}/LICENSE")
	set(CPACK_PACKAGE_VERSION_MAJOR "${PROJECT_VERSION_MAJOR}")
	set(CPACK_PACKAGE_VERSION_MINOR "${PROJECT_VERSION_MINOR}")
	set(CPACK_PACKAGE_VERSION_PATCH "${PROJECT_VERSION_PATCH}")
	set(CPACK_PACKAGE_FILE_NAME "${PROJECT_NAME}_${PROJECT_VERSION}-$ENV{DIST}")
	set(CPACK_STRIP_FILES TRUE)
	set(CPACK_DEBIAN_PACKAGE_HOMEPAGE "https://openbangla.github.io/")
	set(CPACK_DEBIAN_PACKAGE_SECTION "utils")
	set(CPACK_DEBIAN_PACKAGE_DEPENDS "ibus (>= 1.5.1)")
	set(CPACK_DEBIAN_PACKAGE_SHLIBDEPS ON)
	set(CPACK_RPM_PACKAGE_LICENSE "GPLv3")
	set(CPACK_RPM_PACKAGE_URL "https://openbangla.github.io/")
	set(CPACK_RPM_PACKAGE_REQUIRES "qt5-qtbase >= 5.9.0, ibus >= 1.5.1, ibus-libs >= 1.5.1, libzstd >= 1.3.3")
	set(CPACK_RPM_PACKAGE_RELEASE_DIST ON)
	# Prevents CPack from generating file conflicts
	set(CPACK_RPM_EXCLUDE_FROM_AUTO_FILELIST_ADDITION "/usr/share/applications")
	list(APPEND CPACK_RPM_EXCLUDE_FROM_AUTO_FILELIST_ADDITION "/usr/share/ibus")
	list(APPEND CPACK_RPM_EXCLUDE_FROM_AUTO_FILELIST_ADDITION "/usr/share/ibus/component")
	list(APPEND CPACK_RPM_EXCLUDE_FROM_AUTO_FILELIST_ADDITION "/usr/share/metainfo")
	list(APPEND CPACK_RPM_EXCLUDE_FROM_AUTO_FILELIST_ADDITION "/usr/share/pixmaps")
	list(APPEND CPACK_RPM_EXCLUDE_FROM_AUTO_FILELIST_ADDITION "/usr/share/icons")
	list(APPEND CPACK_RPM_EXCLUDE_FROM_AUTO_FILELIST_ADDITION "/usr/share/icons/hicolor")
	list(APPEND CPACK_RPM_EXCLUDE_FROM_AUTO_FILELIST_ADDITION "/usr/share/icons/hicolor/1024x1024")
	list(APPEND CPACK_RPM_EXCLUDE_FROM_AUTO_FILELIST_ADDITION "/usr/share/icons/hicolor/1024x1024/apps")
	list(APPEND CPACK_RPM_EXCLUDE_FROM_AUTO_FILELIST_ADDITION "/usr/share/icons/hicolor/128x128")
	list(APPEND CPACK_RPM_EXCLUDE_FROM_AUTO_FILELIST_ADDITION "/usr/share/icons/hicolor/128x128/apps")
	list(APPEND CPACK_RPM_EXCLUDE_FROM_AUTO_FILELIST_ADDITION "/usr/share/icons/hicolor/16x16")
	list(APPEND CPACK_RPM_EXCLUDE_FROM_AUTO_FILELIST_ADDITION "/usr/share/icons/hicolor/16x16/apps")
	list(APPEND CPACK_RPM_EXCLUDE_FROM_AUTO_FILELIST_ADDITION "/usr/share/icons/hicolor/32x32")
	list(APPEND CPACK_RPM_EXCLUDE_FROM_AUTO_FILELIST_ADDITION "/usr/share/icons/hicolor/32x32/apps")
	list(APPEND CPACK_RPM_EXCLUDE_FROM_AUTO_FILELIST_ADDITION "/usr/share/icons/hicolor/48x48")
	list(APPEND CPACK_RPM_EXCLUDE_FROM_AUTO_FILELIST_ADDITION "/usr/share/icons/hicolor/48x48/apps")
	list(APPEND CPACK_RPM_EXCLUDE_FROM_AUTO_FILELIST_ADDITION "/usr/share/icons/hicolor/512x512")
	list(APPEND CPACK_RPM_EXCLUDE_FROM_AUTO_FILELIST_ADDITION "/usr/share/icons/hicolor/512x512/apps")
	include(CPack)

Thanks!

gunnarhj · 2021-04-22T06:10:51Z

gunnarhj
Apr 22, 2021

How about introducing a compile-time variable that will control the compilation and linking of the 3rdParty code for now?

I think that would be a big step forward.

That code is used for update checking currently. I will remove this functionality when OpenBangla Keyboard gets included in the package repository and seamless update process is ensured.

Actually the 3rd party code will need to be excluded from the source tarball used in Debian, if we want OpenBangla Keyboard to enter Debain's "main" respective Ubuntu's "universe". It would be possible to do that manually when packaging the thing, but it would be more convenient (for me, at least) if coming upstream releases could provide source tarballs without that code.

Also, some expectation management since you mention "seamless update process": Both Debian and Ubuntu work with stable releases. In short it means that software versions at freeze before respective release day stay at those versions. Updates to latest upstream happens in respective development version. As regards supported releases, updates are usually limited to security fixes and targeted important bug fixes. My impression is that OpenBangla Keyboard is a mature enough software to fit in that model.

I have made some attempts to build openbangla-keyboard using the Debian tools, but stumbled upon some difficulties.

First I noticed that the 2.0.0 release source code tarball does not include that riti thing. It seems to me as if the code from the riti subproject should be included in the source tarball. To move forward I simply cloned the current OpenBangla-Keybard repo recursively and used it to create a .tar.gz archive manually.

Then the build succeeded locally, but next problem was this Debian complaint:

E: openbangla-keyboard: arch-dependent-file-in-usr-share usr/share/openbangla-keyboard/ibus-openbangla
E: openbangla-keyboard: arch-dependent-file-in-usr-share usr/share/openbangla-keyboard/openbangla-gui

further explained at this page. For that I have proposed PR #208 as a solution, though.

Finally I tried to build in a Launchpad PPA, which is similar to the build environment used in Ubuntu (and basically also Debian). It failed, and I think the relevant part of the buildlog is this part:

/usr/lib/qt5/bin/uic -o /<<BUILDDIR>>/openbangla-keyboard-2.0.0+git20210409/obj-x86_64-linux-gnu/src/frontend/3rdParty/3rdParty_autogen/include/ui_Downloader.h /<<BUILDDIR>>/openbangla-keyboard-2.0.0+git20210409/src/frontend/3rdParty/Downloader.ui
    Updating crates.io index
warning: spurious network error (2 tries remaining): failed to resolve address for github.com: Name or service not known; class=Net (12)
warning: spurious network error (1 tries remaining): failed to resolve address for github.com: Name or service not known; class=Net (12)
error: failed to get `edit-distance` as a dependency of package `riti v0.1.0 (/<<BUILDDIR>>/openbangla-keyboard-2.0.0+git20210409/src/engine/riti)`

Caused by:
  failed to load source for dependency `edit-distance`

Caused by:
  Unable to update registry `https://github.com/rust-lang/crates.io-index`

Caused by:
  failed to fetch `https://github.com/rust-lang/crates.io-index`

Caused by:
  network failure seems to have happened
  if a proxy or similar is necessary `net.git-fetch-with-cli` may help here
  https://doc.rust-lang.org/cargo/reference/config.html#netgit-fetch-with-cli

Caused by:
  failed to resolve address for github.com: Name or service not known; class=Net (12)
make[3]: *** [src/engine/riti/CMakeFiles/riti_target.dir/build.make:102: src/engine/riti/x86_64-unknown-linux-gnu/debug/libriti.a] Error 101
make[3]: Leaving directory '/<<BUILDDIR>>/openbangla-keyboard-2.0.0+git20210409/obj-x86_64-linux-gnu'
make[2]: *** [CMakeFiles/Makefile2:275: src/engine/riti/CMakeFiles/riti_target.dir/all] Error 2

Why does it want to fetch something at github.com when compiling? Will the problem go away if the 3rd party code is decoupled from the build?

In the PPA you can also see the preliminary packaging files I wrote. Would be good if you could take a look at e.g. the control file (build dependencies) and the copyright file.

0 replies

mominul · 2021-04-22T11:09:18Z

mominul
Apr 22, 2021
Maintainer Author

Updates to latest upstream happens in respective development version.

So we can expect a new version of OBK to be accepted in the repository if the distro development is not in a 'freeze' state?

First I noticed that the 2.0.0 release source code tarball does not include that riti thing. It seems to me as if the code from the riti subproject should be included in the source tarball.

That's a bug of Github of not including submodules in the source tarball. We have to apply a patch in our CI workflow to deploy a source tarball with submodule from Github Actions.

Why does it want to fetch something at github.com when compiling? Will the problem go away if the 3rd party code is decoupled from the build?

Actually, that's not a problem of 3rd party code. Instead, it is arising when compiling the riti library which is written in Rust and uses Cargo package manager. The riti library depends on other libraries(crates) which are downloaded from crates.io repository while the compilation process. Is there any way available to lift the network access restriction of Launchpad? Otherwise, we will have to vendor the dependencies locally, which will require more changes while creating a source tarball. 😞

In the PPA you can also see the preliminary packaging files I wrote. Would be good if you could take a look at e.g. the control file (build dependencies) and the copyright file.

Looks great!

Thanks again!

0 replies

gunnarhj · 2021-04-22T12:21:47Z

gunnarhj
Apr 22, 2021

So we can expect a new version of OBK to be accepted in the repository if the distro development is not in a 'freeze' state?

Yes, normally. But please note that Debian is run by volunteers, so it may lag. For now I will probably handle it if the package gets accepted, but I won't be there forever...

That's a bug of Github of not including submodules in the source tarball. We have to apply a patch in our CI workflow to deploy a source tarball with submodule from Github Actions.

Ack.

Actually, that's not a problem of 3rd party code. Instead, it is arising when compiling the riti library which is written in Rust and uses Cargo package manager. The riti library depends on other libraries(crates) which are downloaded from crates.io repository while the compilation process. Is there any way available to lift the network access restriction of Launchpad?

I have never seen an example of downloading external build deps as a step in the build process, and can't tell if it's possible. I can ask. But in that case it will need to be possible both on Debian and Launchpad.

Otherwise, we will have to vendor the dependencies locally, which will require more changes while creating a source tarball. 😞

Possibly they are already in the Debian/Ubuntu archives. This one looks promising, don't you think?

Edit:

Or this. The Debian archive seems to include some interesting things, but I would appreciate your help to pick the most promising packages to pull as build dependencies.

0 replies

gunnarhj · 2021-04-23T20:03:58Z

gunnarhj
Apr 23, 2021

@mominul: I added librust-crates-io-dev to the build depends and uploaded to the PPA again. Didn't help. Maybe there is more into it, or maybe it's simply not possible. Personally I fear the latter. Debian takes pains to only provide truly free software. Allowing software being downloaded from 3rd party repos during build would compromise the integrity of the Debian archive.

There is a Debian Rust packaging team, and they have a mailing list. Possibly they are able to provide guidance as regards how to go about the crates.io dependencies. If you think it's worth it, you'd better ask a question yourself, since you — unlike me — know how to formulate the question. 😉

1 reply

mominul Apr 24, 2021
Maintainer Author

Didn't help. Maybe there is more into it, or maybe it's simply not possible.

This project is a C++/Rust hybrid so I fear that there might be no simple tool available to solve this problem. 😞 Please give me time, I'll try to make a source tarball with all the dependencies right in place. So that no network access might be needed during the build phase.

gunnarhj · 2021-04-24T20:56:08Z

gunnarhj
Apr 24, 2021

I'll try to make a source tarball with all the dependencies right in place. So that no network access might be needed during the build phase.

Ok, good. That's probably the safest path.

5 replies

mominul Apr 29, 2021
Maintainer Author

I have created a source archive with all the dependencies right in place:
source-vendored.tar.gz

Can you try building with this tarball in launchpad?

gunnarhj Apr 30, 2021

Progress! :) It built successfully on amd64 but failed on other architectures.

   Compiling libc v0.2.94
error[E0463]: can't find crate for `core`
  |
  = note: the `x86_64-unknown-linux-gnu` target may not be installed

Maybe the failures are caused by some kind of architecture confusion. Please see the PPA and check out the buildlogs yourself.

mominul Apr 30, 2021
Maintainer Author

Here is a new source-vendored.tar.gz with the fix. Can you try again, please?

gunnarhj Apr 30, 2021

Success. 😄 It built fine on the 5 architectures I have access to via a Launchpad PPA, so whatever change you made seems to be it.

mominul May 1, 2021
Maintainer Author

Great! 🎉 The fix was this, it was a bug in the CMake module to incorrectly assumed Rust compiler target.

I noticed that the PPA was building for the upcoming Ubuntu Impish release. Can we make the PPA build packages for other Ubuntu versions(18.04, 20.04, etc) also? I'd like to distribute OpenBangla Keyboard through the PPA if possible, as it'd be very easy for the users to install the package from a PPA.

gunnarhj · 2021-05-02T09:58:43Z

gunnarhj
May 2, 2021

@mominul wrote:

Can we make the PPA build packages for other Ubuntu versions(18.04, 20.04, etc) also? I'd like to distribute OpenBangla Keyboard through the PPA if possible, as it'd be very easy for the users to install the package from a PPA.

Yes, that would be possible. But I have another idea, which is probably better.

I would like to see you make a new release soon with the new kind of complete tarball and including the commits in the develop branch we have already talked about. Then I could take the following steps:

Submit OBK to Debian for review. As already mentioned we can expect it to wait in the queue for a while.
Submit the same OBK source to Ubuntu (impish) officially. That approval process is much faster.
Once in impish, I could backport OBK to 18.04, 20.04 and 21.04.

Once in -backports, users would be able to just do:

sudo apt install openbangla-keyboard

without a need to add a PPA.

0 replies

gunnarhj · 2021-05-04T11:54:28Z

gunnarhj
May 4, 2021

I would like to see you make a new release soon with the new kind of complete tarball and including the commits in the develop branch we have already talked about. Then I could take the following steps:

Submit OBK to Debian for review. As already mentioned we can expect it to wait in the queue for a while.

Submit the same OBK source to Ubuntu (impish) officially. That approval process is much faster.

Once in impish, I could backport OBK to 18.04, 20.04 and 21.04.

@mominul: Are we agreed on that plan? In that case the ball is in your court, and I just await your next move.

1 reply

mominul May 4, 2021
Maintainer Author

Yeah, we agree! I'll try to make a new release within a few weeks. 🚀 Sorry for replying late!

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Regarding the inclusion of OpenBangla Keyboard into Debian package repository #207

{{title}}

Replies: 7 comments 7 replies

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Regarding the inclusion of OpenBangla Keyboard into Debian package repository #207

mominul Apr 19, 2021 Maintainer

Replies: 7 comments · 7 replies

mominul Apr 22, 2021 Maintainer Author

mominul Apr 24, 2021 Maintainer Author

mominul Apr 29, 2021 Maintainer Author

mominul Apr 30, 2021 Maintainer Author

mominul May 1, 2021 Maintainer Author

mominul May 4, 2021 Maintainer Author

mominul
Apr 19, 2021
Maintainer

Replies: 7 comments 7 replies

mominul
Apr 22, 2021
Maintainer Author

mominul Apr 24, 2021
Maintainer Author

mominul Apr 29, 2021
Maintainer Author

mominul Apr 30, 2021
Maintainer Author

mominul May 1, 2021
Maintainer Author

mominul May 4, 2021
Maintainer Author