Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[17.0][MIG] connector + test_connector: Migration to 17.0 #476

Merged
merged 796 commits into from
Dec 11, 2023

Conversation

nguyenminhchien
Copy link

@nguyenminhchien nguyenminhchien commented Nov 27, 2023

oca-transbot and others added 30 commits November 27, 2023 10:01
Instead of `ir.module.module`. Simplifies as we never hit the database nor have
to maintain a cache.

Also, it fixes an issue when we run tests directly at the installation of a
module, during the tests the previous implementation was considering the module
as uninstalled instead of installed.

Change proposed @bealdav. Thanks!

Fixes OCA#85
This context manager allows to execute a job function synchronously when it
should normally have been delayed in a asynchronous job.

This is really useful for writing tests that check the flow of a
synchronisation without having to deal with the jobs execution mechanisms.

It is used in the OCA/connector-magento project and ought to be used by other
projects.
This Postgres feature is invaluable when dealing with synchronisations,
especially when importing concurrently records from a system.

When we export a record, we are able to acquire a lock on the exported record
to prevent 2 jobs to export it at the same time. This is different when we
import a record for the first time and with several jobs running in parallel,
chances are high that 2 jobs will import the same record at the same moment.

The Postgres advisory lock comes handy there for they allow to acquire an
application lock.  Usually we'll acquire the lock at the beginning of an import
(beginning of ``Importer.run()``) and we'll throw a ``RetryableJobError`` if
the lock cannot be acquired so the job is retried later. The lock will remain
in place until the end of the transaction.

Example:
 - Job 1 imports Partner A
 - Job 2 imports Partner B
 - Partner A has a category X which happens not to exist yet
 - Partner B has a category X which happens not to exist yet
 - Job 1 import category X as a dependency
 - Job 2 import category X as a dependency

Since both jobs are executed concurrently, they both create a record for category X.
With this lock:

 - Job 1 imports Partner A, it puts a lock for this partner
 - Job 2 imports Partner B, it puts a lock for this partner
 - Partner A has a category X which happens not to exist yet
 - Partner B has a category X which happens not to exist yet
 - Job 1 import category X as a dependency, it puts a lock for this category
 - Job 2 import category X as a dependency, try to put a lock but can't, Job 2
   is retried later, and when it is retried, it sees the category X created by
   Job 1

See http://topopps.com/implementing-postgres-advisory-locks/ for the article
where I learned about the computation of the hash for this purpose.
It is no longer a context manager, because we would expect the lock to be
released at the end of the 'with' statement but it lasts until the end of the
transaction.
Commit c97ebbd was a frontport from 7.0.
However, openerp.sql_db.dsn was changed in 8.0 in that it now returns a
tuple. Extract the db_name from the returned tuple.
The first item contains the name of the DB such as 'odoo_db' and the
second item contains 'user=gbaconnier dbname=odoo_db' which is what
expects psycopg2.connect.
This is a temporary fix. In version 4.0,
dbfilter will be completely ignored by connector.
When PG is localized, error messages are not ascii and jobs are not postponed. Instead they are failed with a 'unicode decode error'.

Note: even if PG ```lc_messages``` is set to en_us.utf8 on a localized system, the error message prefixes (such as ```DETAILS:```) are still localized.
This is a safeguard to prevent someone to write arbitrary code in jobs.
Builtin types and datetime/timedelta are allowed in job arguments, and a
new function 'whitelist_unpickle_global' allows to register new objects
if needed.
lmignon and others added 21 commits December 11, 2023 11:10
removes unknown fields from backend creation args
Currently translated at 43.7% (7 of 16 strings)

Translation: connector-16.0/connector-16.0-test_connector
Translate-URL: https://translation.odoo-community.org/projects/connector-16-0/connector-16-0-test_connector/it/
Currently translated at 43.7% (7 of 16 strings)

Translation: connector-16.0/connector-16.0-test_connector
Translate-URL: https://translation.odoo-community.org/projects/connector-16-0/connector-16-0-test_connector/sl/
Currently translated at 100.0% (16 of 16 strings)

Translation: connector-16.0/connector-16.0-test_connector
Translate-URL: https://translation.odoo-community.org/projects/connector-16-0/connector-16-0-test_connector/es/
@nguyenminhchien
Copy link
Author

@nguyenminhchien all the unreleased dependencies have been merged, can you update the PR?

@gurneyalex updated.

@simahawk
Copy link
Contributor

/ocabot merge nobump

@OCA-git-bot
Copy link
Contributor

On my way to merge this fine PR!
Prepared branch 17.0-ocabot-merge-pr-476-by-simahawk-bump-nobump, awaiting test results.

@OCA-git-bot OCA-git-bot merged commit ef3f94a into OCA:17.0 Dec 11, 2023
5 checks passed
@OCA-git-bot
Copy link
Contributor

Congratulations, your PR was merged at 386df96. Thanks a lot for contributing to OCA. ❤️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet