Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom epmd module - simple solution #4179

Merged
merged 15 commits into from
Nov 30, 2023
Merged

Custom epmd module - simple solution #4179

merged 15 commits into from
Nov 30, 2023

Conversation

arcusfelis
Copy link
Contributor

@arcusfelis arcusfelis commented Nov 24, 2023

This PR addresses "previous PR #4174, but with simplified".

Sometimes, it could take 30 seconds for DNS record to become resolvable in k8s.
In this time, a node could try establishing outgoing erlang distribution connections but could not accept incoming erlang distribution connections. But, global tries to maintain a fully connected mesh between nodes. This means that

new node connects to the cluster
other nodes try to connect to that new node (coordinated by global)
they get nxdomain error inside net_kernel and global gets pang
global gets pang (and nodedown), asks other nodes to disconnect from this new node:
 when=2023-11-02T14:06:42.939527+00:00 level=warning pid=<0.59.0> at=: unstructured_log="'global' at node '[email protected]' disconnected node '[email protected]' in order to prevent overlapping partitions"
when=2023-11-02T14:06:42.936864+00:00 level=warning pid=<0.1310.0> at=: unstructured_log="'global' at '[email protected]' failed to connect to '[email protected]'\

Proposed changes include:

  • mongoose_node_address module and gen_server are gone.
  • No retries. We wait for get_nodes call to finish instead.
  • No extra ETS table - get value from the disco state instead (disco is only blocking for get_nodes call. Join/ping calls are async. So should be safe).
  • No connect.

Requires esl/cets#42 (merged)
Requires esl/cets#43 (merged)

@mongoose-im

This comment was marked as outdated.

Copy link

codecov bot commented Nov 24, 2023

Codecov Report

Attention: 14 lines in your changes are missing coverage. Please review.

Comparison is base (07c2263) 84.03% compared to head (1627994) 84.00%.
Report is 13 commits behind head on master.

Files Patch % Lines
src/mongoose_epmd.erl 55.17% 13 Missing ⚠️
src/mongoose_cluster_id.erl 95.45% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4179      +/-   ##
==========================================
- Coverage   84.03%   84.00%   -0.04%     
==========================================
  Files         564      565       +1     
  Lines       34064    34117      +53     
==========================================
+ Hits        28625    28659      +34     
- Misses       5439     5458      +19     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

…cluster_id

Interrupt other waiting calls in mongoose_cluster_id

Report slow wait_for_rdbms call

Report slow wait_for_any_backend

Use exit with shutdown reason to stop processes in mongoose_cluster_id
Fix query specific test for cets

Reorder columns to match table, insert, update and select queries

Fix graphql_cets_SUITE test
Function address_please waits for get_nodes, gets backend_state,
tries to use IP

Add tests
@mongoose-im
Copy link
Collaborator

mongoose-im commented Nov 28, 2023

elasticsearch_and_cassandra_26 / elasticsearch_and_cassandra_mnesia / 09a4adb
Reports root/ big
OK: 369 / Failed: 0 / User-skipped: 38 / Auto-skipped: 0


small_tests_25 / small_tests / 09a4adb
Reports root / small


small_tests_26 / small_tests / 09a4adb
Reports root / small


small_tests_26_arm64 / small_tests / 09a4adb
Reports root / small


ldap_mnesia_25 / ldap_mnesia / 09a4adb
Reports root/ big
OK: 2311 / Failed: 0 / User-skipped: 872 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_26 / pgsql_mnesia / 09a4adb
Reports root/ big
OK: 4283 / Failed: 0 / User-skipped: 112 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / 09a4adb
Reports root/ big
OK: 4283 / Failed: 0 / User-skipped: 112 / Auto-skipped: 0


ldap_mnesia_26 / ldap_mnesia / 09a4adb
Reports root/ big
OK: 2311 / Failed: 0 / User-skipped: 872 / Auto-skipped: 0


dynamic_domains_mysql_redis_26 / mysql_redis / 09a4adb
Reports root/ big
OK: 4251 / Failed: 0 / User-skipped: 144 / Auto-skipped: 0


internal_mnesia_26 / internal_mnesia / 09a4adb
Reports root/ big
OK: 2461 / Failed: 0 / User-skipped: 722 / Auto-skipped: 0


pgsql_cets_26 / pgsql_cets / 09a4adb
Reports root/ big
OK: 4332 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0


pgsql_mnesia_25 / pgsql_mnesia / 09a4adb
Reports root/ big
OK: 4672 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0


dynamic_domains_mssql_mnesia_26 / odbc_mssql_mnesia / 09a4adb
Reports root/ big
OK: 4280 / Failed: 0 / User-skipped: 115 / Auto-skipped: 0


mysql_redis_26 / mysql_redis / 09a4adb
Reports root/ big
OK: 4652 / Failed: 0 / User-skipped: 139 / Auto-skipped: 0


mssql_mnesia_26 / odbc_mssql_mnesia / 09a4adb
Reports root/ big
OK: 4669 / Failed: 0 / User-skipped: 122 / Auto-skipped: 0


pgsql_mnesia_26 / pgsql_mnesia / 09a4adb
Reports root/ big
OK: 4672 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0

@mongoose-im

This comment was marked as resolved.

@mongoose-im
Copy link
Collaborator

mongoose-im commented Nov 28, 2023

elasticsearch_and_cassandra_26 / elasticsearch_and_cassandra_mnesia / 1247a6b
Reports root/ big
OK: 369 / Failed: 0 / User-skipped: 38 / Auto-skipped: 0


small_tests_25 / small_tests / 1247a6b
Reports root / small


small_tests_26 / small_tests / 1247a6b
Reports root / small


small_tests_26_arm64 / small_tests / 1247a6b
Reports root / small


ldap_mnesia_25 / ldap_mnesia / 1247a6b
Reports root/ big
OK: 2311 / Failed: 0 / User-skipped: 872 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / 1247a6b
Reports root/ big
OK: 4283 / Failed: 0 / User-skipped: 112 / Auto-skipped: 0


ldap_mnesia_26 / ldap_mnesia / 1247a6b
Reports root/ big
OK: 2311 / Failed: 0 / User-skipped: 872 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_26 / pgsql_mnesia / 1247a6b
Reports root/ big
OK: 4283 / Failed: 0 / User-skipped: 112 / Auto-skipped: 0


dynamic_domains_mysql_redis_26 / mysql_redis / 1247a6b
Reports root/ big
OK: 4251 / Failed: 0 / User-skipped: 144 / Auto-skipped: 0


dynamic_domains_mssql_mnesia_26 / odbc_mssql_mnesia / 1247a6b
Reports root/ big
OK: 4280 / Failed: 0 / User-skipped: 115 / Auto-skipped: 0


internal_mnesia_26 / internal_mnesia / 1247a6b
Reports root/ big
OK: 2461 / Failed: 0 / User-skipped: 722 / Auto-skipped: 0


pgsql_mnesia_25 / pgsql_mnesia / 1247a6b
Reports root/ big
OK: 4672 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0


pgsql_cets_26 / pgsql_cets / 1247a6b
Reports root/ big
OK: 4332 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0


mssql_mnesia_26 / odbc_mssql_mnesia / 1247a6b
Reports root/ big
OK: 4669 / Failed: 0 / User-skipped: 122 / Auto-skipped: 0


pgsql_mnesia_26 / pgsql_mnesia / 1247a6b
Reports root/ big
OK: 4671 / Failed: 1 / User-skipped: 119 / Auto-skipped: 0

dynamic_domains_SUITE:with_mod_dynamic_domains_test:packet_handling_for_subdomain
{error,
  {{badrpc,
     {'EXIT',
       {timeout,
         [{meck_proc,wait,6,
            [{file,
               "/home/circleci/project/_build/default/lib/meck/src/meck_proc.erl"},
             {line,171}]},
          {meck,wait,5,[]}]}}},
   [{distributed_helper,rpc,
      [#{node => mongooseim@localhost},
       meck,wait,
       [3,mod_dynamic_domains_test,process_packet,5,500]],
      [{file,
         "/home/circleci/project/big_tests/tests/distributed_helper.erl"},
       {line,140}]},
    {dynamic_domains_SUITE,'-packet_handling_for_subdomain/1-fun-3-',1,
      [{file,
         "/home/circleci/project/big_tests/tests/dynamic_domains_SUITE.erl"},
       {line,113}]},
    {escalus_story,story,4,
      [{file,
         "/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
       {line,72}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1302}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1234}]}]}}

Report log


mysql_redis_26 / mysql_redis / 1247a6b
Reports root/ big
OK: 4651 / Failed: 1 / User-skipped: 139 / Auto-skipped: 0

carboncopy_SUITE:one2one:dropped_client_doesnt_create_duplicate_carbons
{error,
  {{badmatch,
     [{xmlel,<<"message">>,
        [{<<"from">>,
        <<"alice_dropped_client_doesnt_create_duplicate_carbons_713@localhost">>},
         {<<"to">>,
        <<"alice_dropped_client_doesnt_create_duplicate_carbons_713@localhost/res2">>},
         {<<"xmlns">>,<<"jabber:client">>},
         {<<"type">>,<<"chat">>}],
        [{xmlel,<<"sent">>,
           [{<<"xmlns">>,<<"urn:xmpp:carbons:2">>}],
           [{xmlel,<<"forwarded">>,
            [{<<"xmlns">>,<<"urn:xmpp:forward:0">>}],
            [{xmlel,<<"message">>,
               [{<<"from">>,
                 <<"alice_dropped_client_doesnt_create_duplicate_carbons_713@localhost/res1">>},
                {<<"type">>,<<"chat">>},
                {<<"to">>,
                 <<"bob_dropped_client_doesnt_create_duplicate_carbons_713@localhost/res1">>},
                {<<"xmlns">>,<<"jabber:client">>}],
               [{xmlel,<<"body">>,[],
                  [{xmlcdata,
                     <<"And pious action">>}]}]}]}]}]}]},
   [{carboncopy_SUITE,
      '-dropped_client_doesnt_create_duplicate_carbons/1-fun-0-',4,
      [{file,
         "/home/circleci/project/big_tests/tests/carboncopy_SUITE.erl"},
       {line,189}]},
    {escalus_story,story,4,
      [{file,
         "/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
       {line,72}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_serv...

Report log


pgsql_mnesia_26 / pgsql_mnesia / 1247a6b
Reports root/ big
OK: 4672 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0


mysql_redis_26 / mysql_redis / 1247a6b
Reports root/ big
OK: 4652 / Failed: 0 / User-skipped: 139 / Auto-skipped: 0

@mongoose-im
Copy link
Collaborator

mongoose-im commented Nov 28, 2023

elasticsearch_and_cassandra_26 / elasticsearch_and_cassandra_mnesia / d37ec35
Reports root/ big
OK: 369 / Failed: 0 / User-skipped: 38 / Auto-skipped: 0


small_tests_25 / small_tests / d37ec35
Reports root / small


small_tests_26 / small_tests / d37ec35
Reports root / small


small_tests_26_arm64 / small_tests / d37ec35
Reports root / small


ldap_mnesia_25 / ldap_mnesia / d37ec35
Reports root/ big
OK: 2311 / Failed: 0 / User-skipped: 873 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / d37ec35
Reports root/ big
OK: 4284 / Failed: 0 / User-skipped: 112 / Auto-skipped: 0


ldap_mnesia_26 / ldap_mnesia / d37ec35
Reports root/ big
OK: 2311 / Failed: 0 / User-skipped: 873 / Auto-skipped: 0


dynamic_domains_mysql_redis_26 / mysql_redis / d37ec35
Reports root/ big
OK: 4252 / Failed: 0 / User-skipped: 144 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_26 / pgsql_mnesia / d37ec35
Reports root/ big
OK: 4284 / Failed: 0 / User-skipped: 112 / Auto-skipped: 0


dynamic_domains_mssql_mnesia_26 / odbc_mssql_mnesia / d37ec35
Reports root/ big
OK: 4281 / Failed: 0 / User-skipped: 115 / Auto-skipped: 0

graphql_muc_light_SUITE:user:user_muc_light:end_per_group
{error,
 {{unregistering_failed,
   {amount,1},
   {unregistered_items,
  [{{<<"_user_change_room_config_errors_989">>,
     [{escalus_event_mgr,<0.22991.0>},
    {tc_name,user_change_room_config_errors},
    {escalus_cleaner,<0.22990.0>},
    {watchdog,<0.22989.0>},
    {muc_light_host,<<"muclight.domain.example.com">>},
    {secondary_muc_light_host,<<"muclight.domain.example.org">>},
    {schema_endpoint,user},
    {{ejabberd_cwd,mongooseim@localhost},
     "/home/circleci/project/_build/mim1/rel/mongooseim"},
    {preset,"odbc_mssql_mnesia"},
    {mim_data_dir,
     "/home/circleci/project/big_tests/tests/graphql_muc_light_SUITE_data"},
    {tc_logfile,
     "https://circleci-mim-results.s3.eu-central-1.amazonaws.com/PR/4179/201097/odbc_mssql_mnesia.26.1.2-amd64/big/ct_run.test%401e31f76627fc.2023-11-28_17.56.41/big_tests.tests.graphql_muc_light_SUITE.logs/run.2023-11-28_18.00.42/graphql_muc_light_suite.user_change_room_config_errors.34179.html"},
    {tc_group_properties,[{name,user_muc_light},parallel]},
    {tc_group_path,[[{name,user}]]},
    {data_dir,
     "/home/circleci/project/big_tests/_build/default/lib/mongoose_tests/ebin/graphql_muc_light_SUITE_data/"},
    {priv_dir,
     "https://circleci-mim-results.s3.eu-central-1.amazonaws.com/PR/4179/201097/odbc_mssql_mnesia.26.1.2-amd64/big/ct_run.test%401e31f76627fc.2023-11-28_17.56.41/big_tests.tests.graphql_muc_light_SUITE.logs/run.2023-11-28_18.00.42/log_private/"},
    {{saved_modules,mongooseim@localhost,<<"test type">>},
     #{mod_vcard =>
      #{matches => 30,
        host => {prefix,<<"vjud.">>},
        search => true,iqdisc => parallel,bac...

Report log


pgsql_cets_26 / pgsql_cets / d37ec35
Reports root/ big
OK: 4332 / Failed: 0 / User-skipped: 120 / Auto-skipped: 0


internal_mnesia_26 / internal_mnesia / d37ec35
Reports root/ big
OK: 2461 / Failed: 0 / User-skipped: 723 / Auto-skipped: 0


pgsql_mnesia_25 / pgsql_mnesia / d37ec35
Reports root/ big
OK: 4673 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0


mysql_redis_26 / mysql_redis / d37ec35
Reports root/ big
OK: 4653 / Failed: 0 / User-skipped: 139 / Auto-skipped: 0


pgsql_mnesia_26 / pgsql_mnesia / d37ec35
Reports root/ big
OK: 4673 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0


mssql_mnesia_26 / odbc_mssql_mnesia / d37ec35
Reports root/ big
OK: 4670 / Failed: 0 / User-skipped: 122 / Auto-skipped: 0

@mongoose-im

This comment was marked as outdated.

@mongoose-im
Copy link
Collaborator

mongoose-im commented Nov 28, 2023

elasticsearch_and_cassandra_26 / elasticsearch_and_cassandra_mnesia / f8452ee
Reports root/ big
OK: 369 / Failed: 0 / User-skipped: 38 / Auto-skipped: 0


small_tests_25 / small_tests / f8452ee
Reports root / small


small_tests_26 / small_tests / f8452ee
Reports root / small


small_tests_26_arm64 / small_tests / f8452ee
Reports root / small


dynamic_domains_mysql_redis_26 / mysql_redis / f8452ee
Reports root/ big
OK: 4253 / Failed: 0 / User-skipped: 144 / Auto-skipped: 0


ldap_mnesia_26 / ldap_mnesia / f8452ee
Reports root/ big
OK: 2311 / Failed: 0 / User-skipped: 874 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / f8452ee
Reports root/ big
OK: 4285 / Failed: 0 / User-skipped: 112 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_26 / pgsql_mnesia / f8452ee
Reports root/ big
OK: 4285 / Failed: 0 / User-skipped: 112 / Auto-skipped: 0


internal_mnesia_26 / internal_mnesia / f8452ee
Reports root/ big
OK: 2461 / Failed: 0 / User-skipped: 724 / Auto-skipped: 0


dynamic_domains_mssql_mnesia_26 / odbc_mssql_mnesia / f8452ee
Reports root/ big
OK: 4282 / Failed: 0 / User-skipped: 115 / Auto-skipped: 0

graphql_muc_light_SUITE:domain_admin:domain_admin_muc_light:end_per_group
{error,
 {{unregistering_failed,
   {amount,1},
   {unregistered_items,
  [{{<<"_domain_admin_create_identified_room_no_permission_1093">>,
     [{escalus_event_mgr,<0.24712.0>},
    {tc_name,domain_admin_create_identified_room_no_permission},
    {escalus_cleaner,<0.24711.0>},
    {watchdog,<0.24710.0>},
    {muc_light_host,<<"muclight.domain.example.com">>},
    {secondary_muc_light_host,<<"muclight.domain.example.org">>},
    {protocol,http},
    {domain_admin,{<<"[email protected]">>,<<"47fe2e9204f8e6fb">>}},
    {schema_endpoint,domain_admin},
    {{ejabberd_cwd,mongooseim@localhost},
     "/home/circleci/project/_build/mim1/rel/mongooseim"},
    {preset,"odbc_mssql_mnesia"},
    {mim_data_dir,
     "/home/circleci/project/big_tests/tests/graphql_muc_light_SUITE_data"},
    {tc_logfile,
     "https://circleci-mim-results.s3.eu-central-1.amazonaws.com/PR/4179/201162/odbc_mssql_mnesia.26.1.2-amd64/big/ct_run.test%40120e9d286647.2023-11-28_19.13.13/big_tests.tests.graphql_muc_light_SUITE.logs/run.2023-11-28_19.17.33/graphql_muc_light_suite.domain_admin_create_identified_room_no_permission.html"},
    {tc_group_properties,[{name,domain_admin_muc_light}]},
    {tc_group_path,[[{name,domain_admin}]]},
    {data_dir,
     "/home/circleci/project/big_tests/_build/default/lib/mongoose_tests/ebin/graphql_muc_light_SUITE_data/"},
    {priv_dir,
     "https://circleci-mim-results.s3.eu-central-1.amazonaws.com/PR/4179/201162/odbc_mssql_mnesia.26.1.2-amd64/big/ct_run.test%40120e9d286647.2023-11-28_19.13.13/big_tests.tests.graphql_muc_light_SUITE.logs/run.2023-11-28_19.17.33/log_private/"},
    {{saved_modules,m...

Report log


mysql_redis_26 / mysql_redis / f8452ee
Reports root/ big
OK: 4653 / Failed: 1 / User-skipped: 139 / Auto-skipped: 0

carboncopy_SUITE:one2one:dropped_client_doesnt_create_duplicate_carbons
{error,
  {{badmatch,
     [{xmlel,<<"message">>,
        [{<<"from">>,
        <<"alice_dropped_client_doesnt_create_duplicate_carbons_592@localhost">>},
         {<<"to">>,
        <<"alice_dropped_client_doesnt_create_duplicate_carbons_592@localhost/res2">>},
         {<<"xmlns">>,<<"jabber:client">>},
         {<<"type">>,<<"chat">>}],
        [{xmlel,<<"sent">>,
           [{<<"xmlns">>,<<"urn:xmpp:carbons:2">>}],
           [{xmlel,<<"forwarded">>,
            [{<<"xmlns">>,<<"urn:xmpp:forward:0">>}],
            [{xmlel,<<"message">>,
               [{<<"from">>,
                 <<"alice_dropped_client_doesnt_create_duplicate_carbons_592@localhost/res1">>},
                {<<"type">>,<<"chat">>},
                {<<"to">>,
                 <<"bob_dropped_client_doesnt_create_duplicate_carbons_592@localhost/res1">>},
                {<<"xmlns">>,<<"jabber:client">>}],
               [{xmlel,<<"body">>,[],
                  [{xmlcdata,
                     <<"And pious action">>}]}]}]}]}]}]},
   [{carboncopy_SUITE,
      '-dropped_client_doesnt_create_duplicate_carbons/1-fun-0-',4,
      [{file,
         "/home/circleci/project/big_tests/tests/carboncopy_SUITE.erl"},
       {line,189}]},
    {escalus_story,story,4,
      [{file,
         "/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
       {line,72}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_serv...

Report log


pgsql_cets_26 / pgsql_cets / f8452ee
Reports root/ big
OK: 4332 / Failed: 0 / User-skipped: 121 / Auto-skipped: 0


pgsql_mnesia_25 / pgsql_mnesia / f8452ee
Reports root/ big
OK: 4674 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0


pgsql_mnesia_26 / pgsql_mnesia / f8452ee
Reports root/ big
OK: 4674 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0


mssql_mnesia_26 / odbc_mssql_mnesia / f8452ee
Reports root/ big
OK: 4671 / Failed: 0 / User-skipped: 122 / Auto-skipped: 0

graphql_muc_light_SUITE:domain_admin:domain_admin_muc_light:end_per_group
{error,
 {{unregistering_failed,
   {amount,1},
   {unregistered_items,
  [{{<<"_admin_invite_user_errors_1112">>,
     [{escalus_event_mgr,<0.24995.0>},
    {tc_name,admin_invite_user_errors},
    {escalus_cleaner,<0.24994.0>},
    {watchdog,<0.24993.0>},
    {muc_light_host,<<"muclight.localhost">>},
    {secondary_muc_light_host,<<"muclight.localhost.bis">>},
    {protocol,http},
    {domain_admin,{<<"admin@localhost">>,<<"6f7091fc4c8c33b3">>}},
    {schema_endpoint,domain_admin},
    {{ejabberd_cwd,mongooseim@localhost},
     "/home/circleci/project/_build/mim1/rel/mongooseim"},
    {preset,"odbc_mssql_mnesia"},
    {mim_data_dir,
     "/home/circleci/project/big_tests/tests/graphql_muc_light_SUITE_data"},
    {tc_logfile,
     "https://circleci-mim-results.s3.eu-central-1.amazonaws.com/PR/4179/201168/odbc_mssql_mnesia.26.1.2-amd64/big/ct_run.test%4060d92cb211b9.2023-11-28_19.13.06/big_tests.tests.graphql_muc_light_SUITE.logs/run.2023-11-28_19.18.16/graphql_muc_light_suite.admin_invite_user_errors.37251.html"},
    {tc_group_properties,[{name,domain_admin_muc_light}]},
    {tc_group_path,[[{name,domain_admin}]]},
    {data_dir,
     "/home/circleci/project/big_tests/_build/default/lib/mongoose_tests/ebin/graphql_muc_light_SUITE_data/"},
    {priv_dir,
     "https://circleci-mim-results.s3.eu-central-1.amazonaws.com/PR/4179/201168/odbc_mssql_mnesia.26.1.2-amd64/big/ct_run.test%4060d92cb211b9.2023-11-28_19.13.06/big_tests.tests.graphql_muc_light_SUITE.logs/run.2023-11-28_19.18.16/log_private/"},
    {{saved_modules,mongooseim@localhost,<<"localhost">>},
     #{mod_vcard =>
      #{matches => 30,
        hos...

Report log


mysql_redis_26 / mysql_redis / f8452ee
Reports root/ big
OK: 4654 / Failed: 0 / User-skipped: 139 / Auto-skipped: 0

@arcusfelis arcusfelis marked this pull request as ready for review November 28, 2023 20:13
Copy link
Member

@chrzaszcz chrzaszcz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's going in the right direction. I added a few comments.

{error, {no_ip_in_db, Node}};
#{Node := Bin} ->
{ok, IP} = inet:parse_address(binary_to_list(Bin)),
case ?MODULE:can_connect(IP) of
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two questions:

  1. Why prefix with ?MODULE? A local call should be enough.
  2. Why try to connect at this point? We are expected to only return the address. Wouldn't the connection just fail later anyway? What are we gaining at the cost of these extra network operations?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • for meck.
  • it fallbacks to resolver if IP in DB is old and non-relevant anymore. There is a chance resolver would get a newer IP.

I probably could remove can_connect, because it was useful when there was code to retry in a loop.

@@ -47,6 +50,48 @@ start() ->
{error, conflict}
end.


%% If RDBMS is available before CETS - it is enough for us to continue
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two questions:

  1. If CETS is not ready, it might fail at set_new_cluster_id, right?
  2. How is this change connected to EPMD?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. here we want to ensure that we read existing cluster ID - so we want to access our another peer (CETS node or RDBMS). If CETS is not ready, and we don't wait for RDBMS - we will just generate a new cluster ID, which we do not want.
  2. It speeds up starting time. We block the whole MIM initialization here, so if we can just unblock after RDBMS returns - it could be faster.

Comment on lines 44 to 45
start_node_address_server(mim()),
start_node_address_server(mim2()),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Forgot to remove?

Comment on lines 55 to 56
stop_node_address_server(mim()),
stop_node_address_server(mim2()),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Forgot to remove?

Comment on lines 328 to 334
start_node_address_server(Node) ->
MFA = {mongoose_node_address, start_link, []},
ChildSpec = #{id => mongoose_node_address, start => MFA, restart => temporary},
rpc(Node, supervisor, start_child, [ejabberd_sup, ChildSpec]).

stop_node_address_server(Node) ->
rpc(Node, supervisor, terminate_child, [ejabberd_sup, mongoose_node_address]).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Forgot to remove?

rpc(mim(), erlang, send, [mongoose_cets_discovery, check]),
Config;
_ ->
{skip, cets_disco_already_running}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is strange, because we will skip cets-specific tests in the cets preset, but run them in other presets. It is unexpected, because the cets preset should be the one for which all CETS functionality is tested. I'd rather have it skipped in other presets than in the CETS one.

@@ -20,3 +20,5 @@

## Tweak GC to run more often
-env ERL_FULLSWEEP_AFTER 2

{{epmd_vm_args}}
Copy link
Member

@chrzaszcz chrzaszcz Nov 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will need to document this option. Maybe it should be enabled by default (but then we need to make sure it doesn't break without CETS).

Or just remove test disco process in case we are not in CETS preset
but still want to test RDBMS logic with all DB backends
@mongoose-im
Copy link
Collaborator

mongoose-im commented Nov 29, 2023

elasticsearch_and_cassandra_26 / elasticsearch_and_cassandra_mnesia / dad1fbf
Reports root/ big
OK: 369 / Failed: 0 / User-skipped: 38 / Auto-skipped: 0


small_tests_25 / small_tests / dad1fbf
Reports root / small


small_tests_26 / small_tests / dad1fbf
Reports root / small


small_tests_26_arm64 / small_tests / dad1fbf
Reports root / small


ldap_mnesia_25 / ldap_mnesia / dad1fbf
Reports root/ big
OK: 2311 / Failed: 0 / User-skipped: 875 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / dad1fbf
Reports root/ big
OK: 4286 / Failed: 0 / User-skipped: 112 / Auto-skipped: 0


ldap_mnesia_26 / ldap_mnesia / dad1fbf
Reports root/ big
OK: 2311 / Failed: 0 / User-skipped: 875 / Auto-skipped: 0


dynamic_domains_mysql_redis_26 / mysql_redis / dad1fbf
Reports root/ big
OK: 4254 / Failed: 0 / User-skipped: 144 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_26 / pgsql_mnesia / dad1fbf
Reports root/ big
OK: 4286 / Failed: 0 / User-skipped: 112 / Auto-skipped: 0


pgsql_cets_26 / pgsql_cets / dad1fbf
Reports root/ big
OK: 4335 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0


pgsql_mnesia_26 / pgsql_mnesia / dad1fbf
Reports root/ big
OK: 4675 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0


mysql_redis_26 / mysql_redis / dad1fbf
Reports root/ big
OK: 4655 / Failed: 0 / User-skipped: 139 / Auto-skipped: 0


dynamic_domains_mssql_mnesia_26 / odbc_mssql_mnesia / dad1fbf
Reports root/ big
OK: 4283 / Failed: 0 / User-skipped: 115 / Auto-skipped: 0


internal_mnesia_26 / internal_mnesia / dad1fbf
Reports root/ big
OK: 2461 / Failed: 0 / User-skipped: 725 / Auto-skipped: 0


pgsql_mnesia_25 / pgsql_mnesia / dad1fbf
Reports root/ big
OK: 4675 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0


mssql_mnesia_26 / odbc_mssql_mnesia / dad1fbf
Reports root/ big
OK: 4671 / Failed: 1 / User-skipped: 122 / Auto-skipped: 0

graphql_muc_light_SUITE:domain_admin:domain_admin_muc_light:admin_create_room_with_unprepped_id
{error,
  {{badmatch,null},
   [{graphql_muc_light_SUITE,admin_create_room_with_unprepped_id,1,
      [{file,
         "/home/circleci/project/big_tests/tests/graphql_muc_light_SUITE.erl"},
       {line,1155}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1302}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1234}]}]}}

Report log


mssql_mnesia_26 / odbc_mssql_mnesia / dad1fbf
Reports root/ big
OK: 4670 / Failed: 2 / User-skipped: 122 / Auto-skipped: 0

graphql_muc_light_SUITE:admin_http:admin_muc_light:end_per_group
{error,
 {{unregistering_failed,
   {amount,1},
   {unregistered_items,
  [{{<<"_admin_send_message_to_room_1042">>,
     [{escalus_event_mgr,<0.23930.0>},
    {tc_name,admin_send_message_to_room},
    {escalus_cleaner,<0.23929.0>},
    {watchdog,<0.23928.0>},
    {muc_light_host,<<"muclight.localhost">>},
    {secondary_muc_light_host,<<"muclight.localhost.bis">>},
    {protocol,http},
    {schema_endpoint,admin},
    {listener_opts,
     #{module => mongoose_graphql_handler,path => "/api/graphql",
       host => "localhost",username => <<"admin">>,
       password => <<"secret">>,schema_endpoint => admin}},
    {{ejabberd_cwd,mongooseim@localhost},
     "/home/circleci/project/_build/mim1/rel/mongooseim"},
    {preset,"odbc_mssql_mnesia"},
    {mim_data_dir,
     "/home/circleci/project/big_tests/tests/graphql_muc_light_SUITE_data"},
    {tc_logfile,
     "https://circleci-mim-results.s3.eu-central-1.amazonaws.com/PR/4179/201208/odbc_mssql_mnesia.26.1.2-amd64/big/ct_run.test%40cfbdc80cac88.2023-11-29_12.04.49/big_tests.tests.graphql_muc_light_SUITE.logs/run.2023-11-29_12.09.24/graphql_muc_light_suite.admin_send_message_to_room.html"},
    {tc_group_properties,[{name,admin_muc_light}]},
    {tc_group_path,[[{name,admin_http}]]},
    {data_dir,
     "/home/circleci/project/big_tests/_build/default/lib/mongoose_tests/ebin/graphql_muc_light_SUITE_data/"},
    {priv_dir,
     "https://circleci-mim-results.s3.eu-central-1.amazonaws.com/PR/4179/201208/odbc_mssql_mnesia.26.1.2-amd64/big/ct_run.test%40cfbdc80cac88.2023-11-29_12.04.49/big_tests.tests.graphql_muc_light_SUITE.logs/run.2023-11-29_12.09.24/log_private/"}...

Report log

graphql_muc_light_SUITE:admin_cli:admin_muc_light:admin_create_room_with_unprepped_id
{error,
  {#{what => invalid_response_code,expected_type => ok,
     response_code => {exit_status,1}},
   [{graphql_helper,assert_response_code,2,
      [{file,"/home/circleci/project/big_tests/tests/graphql_helper.erl"},
       {line,256}]},
    {graphql_helper,get_ok_value,2,
      [{file,"/home/circleci/project/big_tests/tests/graphql_helper.erl"},
       {line,239}]},
    {graphql_muc_light_SUITE,admin_create_room_with_unprepped_id,1,
      [{file,
         "/home/circleci/project/big_tests/tests/graphql_muc_light_SUITE.erl"},
       {line,1156}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1302}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1234}]}]}}

Report log

graphql_muc_light_SUITE:domain_admin:domain_admin_muc_light:admin_create_room_with_unprepped_id
{error,
  {{badmatch,null},
   [{graphql_muc_light_SUITE,admin_create_room_with_unprepped_id,1,
      [{file,
         "/home/circleci/project/big_tests/tests/graphql_muc_light_SUITE.erl"},
       {line,1155}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1302}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1234}]}]}}

Report log


mssql_mnesia_26 / odbc_mssql_mnesia / dad1fbf
Reports root/ big
OK: 4672 / Failed: 0 / User-skipped: 122 / Auto-skipped: 0

muc_light_http_api_SUITE:end_per_suite
{error,
 {{unregistering_failed,
   {amount,1},
   {unregistered_items,
  [{{<<"_send_message_errors_2716">>,
     [{escalus_event_mgr,<0.72031.0>},
    {tc_name,send_message_errors},
    {escalus_cleaner,<0.72030.0>},
    {watchdog,<0.72029.0>},
    {preset,"odbc_mssql_mnesia"},
    {mim_data_dir,
     "/home/circleci/project/big_tests/tests/muc_light_http_api_SUITE_data"},
    {tc_logfile,
     "https://circleci-mim-results.s3.eu-central-1.amazonaws.com/PR/4179/201209/odbc_mssql_mnesia.26.1.2-amd64/big/ct_run.test%4016d5a249bab2.2023-11-29_12.41.16/big_tests.tests.muc_light_http_api_SUITE.logs/run.2023-11-29_12.56.24/muc_light_http_api_suite.send_message_errors.93188.html"},
    {tc_group_properties,[{name,negative},parallel]},
    {tc_group_path,[]},
    {data_dir,
     "/home/circleci/project/big_tests/_build/default/lib/mongoose_tests/ebin/muc_light_http_api_SUITE_data/"},
    {priv_dir,
     "https://circleci-mim-results.s3.eu-central-1.amazonaws.com/PR/4179/201209/odbc_mssql_mnesia.26.1.2-amd64/big/ct_run.test%4016d5a249bab2.2023-11-29_12.41.16/big_tests.tests.muc_light_http_api_SUITE.logs/run.2023-11-29_12.56.24/log_private/"},
    {{saved_modules,mongooseim@localhost,<<"localhost">>},
     #{mod_vcard =>
      #{matches => 30,
        host => {prefix,<<"vjud.">>},
        search => true,iqdisc => parallel,backend => rdbms},
       mod_roster =>
      #{iqdisc => one_queue,backend => rdbms,versioning => false,
        store_current_id => false},
       mod_sic => #{iqdisc => one_queue},
       mod_register =>
      #{access => register,iqdisc => one_queue,
        registration_watchers => [],password_...

Report log

@arcusfelis
Copy link
Contributor Author

Version without can_connect triggers global's overlap protection during updates.
I will invistigate.

@mongoose-im
Copy link
Collaborator

mongoose-im commented Nov 30, 2023

elasticsearch_and_cassandra_26 / elasticsearch_and_cassandra_mnesia / 1627994
Reports root/ big
OK: 369 / Failed: 0 / User-skipped: 38 / Auto-skipped: 0


small_tests_25 / small_tests / 1627994
Reports root / small


small_tests_26 / small_tests / 1627994
Reports root / small


small_tests_26_arm64 / small_tests / 1627994
Reports root / small


ldap_mnesia_25 / ldap_mnesia / 1627994
Reports root/ big
OK: 2311 / Failed: 0 / User-skipped: 875 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / 1627994
Reports root/ big
OK: 4286 / Failed: 0 / User-skipped: 112 / Auto-skipped: 0


ldap_mnesia_26 / ldap_mnesia / 1627994
Reports root/ big
OK: 2311 / Failed: 0 / User-skipped: 875 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_26 / pgsql_mnesia / 1627994
Reports root/ big
OK: 4286 / Failed: 0 / User-skipped: 112 / Auto-skipped: 0


pgsql_mnesia_25 / pgsql_mnesia / 1627994
Reports root/ big
OK: 4675 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0


dynamic_domains_mysql_redis_26 / mysql_redis / 1627994
Reports root/ big
OK: 4254 / Failed: 0 / User-skipped: 144 / Auto-skipped: 0


dynamic_domains_mssql_mnesia_26 / odbc_mssql_mnesia / 1627994
Reports root/ big
OK: 4283 / Failed: 0 / User-skipped: 115 / Auto-skipped: 0


pgsql_cets_26 / pgsql_cets / 1627994
Reports root/ big
OK: 4335 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0


internal_mnesia_26 / internal_mnesia / 1627994
Reports root/ big
OK: 2461 / Failed: 0 / User-skipped: 725 / Auto-skipped: 0


mysql_redis_26 / mysql_redis / 1627994
Reports root/ big
OK: 4655 / Failed: 0 / User-skipped: 139 / Auto-skipped: 0


pgsql_mnesia_26 / pgsql_mnesia / 1627994
Reports root/ big
OK: 4675 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0


mssql_mnesia_26 / odbc_mssql_mnesia / 1627994
Reports root/ big
OK: 4672 / Failed: 0 / User-skipped: 122 / Auto-skipped: 0

Copy link
Member

@chrzaszcz chrzaszcz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. I added minor comments. Before the release we need to add more information about when to use mongoose_epmd - for example we can put it in the CETS tutorial.

State1 = disco_init(mim(), Opts1),
State2 = disco_init(mim2(), Opts2),
{{ok, _Nodes1_2}, State1_2} = disco_get_nodes(mim(), State1),
{{ok, _Nodes2_2}, State2_2} = disco_get_nodes(mim2(), State2),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should already see the entry from mim() at this point, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right, I guess

@@ -1,5 +1,6 @@
%% vm.args
{node_name, "mongooseim3@localhost"}.
{epmd_module, "mongoose_epmd"}.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this used in any test?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we check that the node could still run using this epmd module, so as a smoke test and to ensure that if no CETS is running, we can still cluster if we set it.

@chrzaszcz chrzaszcz merged commit 44b57e6 into master Nov 30, 2023
3 of 4 checks passed
@chrzaszcz chrzaszcz deleted the custom-epmd-simple branch November 30, 2023 15:10
@chrzaszcz chrzaszcz added this to the 6.2.0 milestone Dec 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants