[New Module] zos_replace #2044

AndreMarcel99 · 2025-04-09T20:41:29Z

SUMMARY

Develop the module zos_replace to generate outputs as the community base on re python library and add options to the user for replace regular strings.

Fixes #1994

ISSUE TYPE

New Module Pull Request

COMPONENT NAME

Module zos_replace with is code flow to open a file on binary mode to avoid any problem with special characters and maintain consistency on the files given by the users.

ADDITIONAL INFORMATION

The test case ds_backup_no_name works locally, but not on the pipeline, work in progress.

richp405

Why not use some of the test entries as examples in the doc? Just a thought.

fernandofloresg

Requested some changes to docs.

fernandofloresg · 2025-04-29T23:14:05Z

plugins/modules/zos_replace.py

+      - When set to C(true), the module creates a backup file or data set.
+      - The backup file name will be returned on either success or failure of
+        module execution such that data can be retrieved.
+      - Use generation data set (GDS) relative positive name. e.g. I(SOME.CREATION(+1)).


I would add "is allowed." at the end here.

Suggested change

- Use generation data set (GDS) relative positive name. e.g. I(SOME.CREATION(+1)).

- Use generation data set (GDS) relative positive name is allowed. e.g. I(SOME.CREATION(+1)).

But since this is for backup option, which is bool, I think this entry should be in backup_name.

fernandofloresg · 2025-04-29T23:18:41Z

plugins/modules/zos_replace.py

+      - The location can be a UNIX System Services (USS) file,
+        PS (sequential data set), member of a PDS or PDSE, PDS, PDSE.
+      - The USS file must be an absolute pathname.
+      - Generation data set (GDS) relative name of generation already


Suggested change

- Generation data set (GDS) relative name of generation already

- It is possible to use a generation data set (GDS) relative name of generation already

fernandofloresg · 2025-04-29T23:20:35Z

plugins/modules/zos_replace.py

+
+
+def resolve_src_name(module, name, results):
+    """Function to solve and validate the exist of the dataset or uss file.


Suggested change

"""Function to solve and validate the exist of the dataset or uss file.

"""Function to resolve and validate the existence of the dataset or uss file.

fernandofloresg · 2025-04-29T23:22:28Z

plugins/modules/zos_replace.py

+
+
+def replace_text(content, regexp, replace):
+    """Function received the test to work on it to find the regexp


Suggested change

"""Function received the test to work on it to find the regexp

"""Function received the text to work on it to find the regexp

fernandofloresg · 2025-04-29T23:23:21Z

plugins/modules/zos_replace.py

+    Parameters
+    ----------
+        content : list
+            The part or full text to be modified


Suggested change

The part or full text to be modified

The partial or complete text to be modified.

Co-authored-by: Alex Moreno <[email protected]>

…nd replace with backref

AndreMarcel99 · 2025-05-16T17:45:01Z

rexemin

Thanks for addressing all comments I've had over the past month, this module looks really good

surendrababuravella

Looks fine

fernandofloresg

Requested few changes, btw can you also add this module name to the github issue templates?

fernandofloresg · 2025-05-26T21:45:49Z

tests/functional/modules/test_zos_replace_func.py

+def set_ds_environment(ansible_zos_module, temp_file, ds_name, ds_type, content):
+    hosts = ansible_zos_module
+    hosts.all.shell(cmd=f"echo \"{content}\" > {temp_file}")
+    hosts.all.zos_data_set(name=ds_name, type=ds_type)


Please use ZOAU utilities with shell module to reduce the module interdependency in tests. This will be useful when migrating to 1.4.0.

plugins/modules/zos_replace.py

Co-authored-by: Fernando Flores <[email protected]>

fernandofloresg

LGTM! Great work on this module, only requested an improvement but this is ready for me. Will see if we can engage a technical writer to review docs content.

AndreMarcel99 · 2025-06-02T21:17:55Z

ddimatos

This looks great @AndreMarcel99 , I really like what you have done. All my comments are minor but should be addressed.

plugins/modules/zos_replace.py

ddimatos · 2025-06-03T17:17:32Z

plugins/modules/zos_replace.py

+      - If I(src) is a data set member and backup_name is not provided, the data set
+        member will be backed up to the same partitioned data set with a randomly generated
+        member name.
+      - If it is a Generation Data Set (GDS), use a relative positive name, e.g., I(SOME.CREATION(+1)).


This is cosmetic bu tif you move this line "If it is a Generation Data Set (GDS), use a relative positive name, e.g., I(SOME.CREATION(+1))." right after the line 48 where you have "If the source is an MVS data set...." then you would be grouping the use case where if it is a certain type, followed then by use cases "it is not provided".

Basically the GDS feels off because its at the bottom after the section "It is not provided"

ddimatos · 2025-06-03T17:19:50Z

plugins/modules/zos_replace.py

+      - If I(src) is a data set member and backup_name is not provided, the data set
+        member will be backed up to the same partitioned data set with a randomly generated
+        member name.
+      - If it is a Generation Data Set (GDS), use a relative positive name, e.g., I(SOME.CREATION(+1)).


For backup_name not provided types USS you cover what happens, you also cover what happens when the backup_name is not provided when the src is a data set member but its not clear to me what happens if the types are either sequential or GDS (as in what happens when not provided for these two types).

plugins/modules/zos_replace.py

ddimatos · 2025-06-03T17:41:27Z

plugins/modules/zos_replace.py

+        requires it to be provided with correct encoding to read the content
+        of a USS file or data set. If this parameter is not provided, this
+        module assumes that USS file or data set is encoded in IBM-1047.
+      - Supported character sets rely on the charset conversion utility (iconv)


I will let you decide, but how would one find the list of character sets supported? We do have doc for this because its to large to list in any module, that doc is here:
https://ibm.github.io/z_ansible_collections_doc/ibm_zos_core/docs/source/character_set.html

And it discusses the iconv -l command that will aid in list the supported encodings for iconv.
I believe the encoding module puts this information in the notes section, it does not have to be in the notes section but I leave this up to you.

ddimatos · 2025-06-03T18:05:26Z

plugins/modules/zos_replace.py

+    default: IBM-1047
+  disable_regex:
+    description:
+      - A list or string that allows the user to specify choices "before", "after", or "regexp" as regular strings instead of regex patterns.


I was a bit confused by this module option, the example cleared it up some, here are my thoughts but also think more needs to be put into this description, maybe when I get to the code I will have a better understanding.

I suggest don't speak in third person "A list or string that allows the user to specify choices" , instead "A list or string that enables you to..." or just "enables....".

Don't say choices when its not really a choice given its type is raw , find another way have the user use only one of these 3 strings "before", "after", or "regexp", see below at my attempt to convey this.

Since your type is raw, what happens if a user does this "disable_regex:"before, after, regexp", which is probably what you want them to think in terms of choices, likely this is not supported so we might need to be a bit more clear on this.

Here is my attempt with not fully understanding this option, it's a bit confusing because I believe (without having looked at code) what this option is doing is disabling the regular expression processing, for example, you want to match this line but not expand the *, "regexp: //SYSPRINT DD SYSOUT=*"

- A list or string that disables the regular expression processing for special characters. - This is useful when the I(regex) to match is a string that contains special characters such as `*` that should not be expanded as a regular expression. - The supported options are "before", "after", or "regexp" as regular strings.

@ddimatos Multi option in this is supported. Maybe a clearer way to say what this module does when this option is given is that treats the fields as literal instead of regex expressions, that way if you have a complex regex that you don't want to waste time escaping or don't have the skills to do so just disable the regex for those fields and the text searched for in the text "as is".

plugins/modules/zos_replace.py

ddimatos · 2025-06-03T18:14:32Z

plugins/modules/zos_replace.py

+RETURN = r"""
+backup_name:
+    description: Name of the backup file or data set that was created.
+    returned: if backup=true


This is a bit confusing, returned if backup=true yet in the documentation it states it will "be returned on either success or failure"

- The backup file name will be returned on either success or failure of module execution such that data can be retrieved.

ddimatos · 2025-06-03T18:17:26Z

plugins/modules/zos_replace.py

+    type: int
+    sample: 5
+msg:
+    description: Error messages from the module


Is msg really used for error messages? I have not read down through the code but does not align to the Ansible doc for a module https://docs.ansible.com/ansible/latest/reference_appendices/common_return_values.html#msg

Co-authored-by: Demetri <[email protected]>

AndreMarcel99 added 20 commits April 8, 2025 09:35

Add first iteration

d086c92

Add replace and logic to replace on destination

4d9ef1c

First version with full functionality

0db79ee

Fix error message

4824dc3

Add extra validation

eac1867

Add documentation and tmphlq variable

45e3abe

Merge branch 'dev' into module/1994/zos_replace

f6362db

First version functional

77bd483

Add import

1fb7023

Merge branch 'dev' into module/1994/zos_replace

8982c5e

Add validation

8889d37

Add validations

56ab986

Add first steps

2f1f95b

Add validation for remove all the line

57d3264

Add new testing

ba125ba

Add for any case before after

c83aa56

Test in real function

c4d193a

Ensure spaces of users are ther

fa1a6ee

Add all base test cases with before and after

7ac5f03

Add full test cases for all basic cases

1b95949

AndreMarcel99 requested review from fernandofloresg, rexemin, richp405 and surendrababuravella April 28, 2025 21:59

AndreMarcel99 added 2 commits April 28, 2025 18:42

Fix for gdg and verify if change the file

37af39d

Add test for backup uss and gdg

0e39ea8

richp405 approved these changes Apr 29, 2025

View reviewed changes

Merge branch 'dev' into module/1994/zos_replace

f698193

fernandofloresg requested changes Apr 30, 2025

View reviewed changes

Add diferent of the module

2e409c1

AndreMarcel99 and others added 3 commits May 16, 2025 10:47

Update plugins/modules/zos_replace.py

b1123e9

Co-authored-by: Alex Moreno <[email protected]>

Add test cases for last cases

438fc71

Add try exception to catch possible miss match and bad use of regex a…

df17bcb

…nd replace with backref

AndreMarcel99 requested a review from rexemin May 16, 2025 16:58

AndreMarcel99 and others added 2 commits May 16, 2025 11:58

Merge branch 'dev' into module/1994/zos_replace

2f843e0

Fix cat properly

ffec4eb

rexemin approved these changes May 20, 2025

View reviewed changes

surendrababuravella approved these changes May 22, 2025

View reviewed changes

fernandofloresg requested changes May 26, 2025

View reviewed changes

AndreMarcel99 and others added 4 commits May 28, 2025 11:31

Update plugins/modules/zos_replace.py

48e9b54

Co-authored-by: Fernando Flores <[email protected]>

Merge branch 'dev' into module/1994/zos_replace

2fb43af

Add multiline regex

804bcaf

Add write one time

a46c28b

AndreMarcel99 requested a review from fernandofloresg May 28, 2025 19:07

Add test case working

bdb99b7

fernandofloresg approved these changes Jun 2, 2025

View reviewed changes

AndreMarcel99 added 2 commits June 2, 2025 11:34

Avoid cost of computation on USS wrting

3d7f15a

Add docs

27f6b43

At to github acctions

d669b52

ddimatos requested changes Jun 3, 2025

View reviewed changes

AndreMarcel99 and others added 7 commits June 3, 2025 20:41

Update plugins/modules/zos_replace.py

2188394

Co-authored-by: Demetri <[email protected]>

Update plugins/modules/zos_replace.py

26cb65f

Co-authored-by: Demetri <[email protected]>

Update plugins/modules/zos_replace.py

8d20cd2

Co-authored-by: Demetri <[email protected]>

Update plugins/modules/zos_replace.py

1d02b47

Co-authored-by: Demetri <[email protected]>

Merge branch 'dev' into module/1994/zos_replace

debd179

Fix documentation

ce62b08

Fix documentation

b1a4f55

AndreMarcel99 requested a review from ddimatos June 4, 2025 17:51

	- Use generation data set (GDS) relative positive name. e.g. I(SOME.CREATION(+1)).
	- Use generation data set (GDS) relative positive name is allowed. e.g. I(SOME.CREATION(+1)).

	- Generation data set (GDS) relative name of generation already
	- It is possible to use a generation data set (GDS) relative name of generation already



		def resolve_src_name(module, name, results):
		"""Function to solve and validate the exist of the dataset or uss file.

	"""Function to solve and validate the exist of the dataset or uss file.
	"""Function to resolve and validate the existence of the dataset or uss file.



		def replace_text(content, regexp, replace):
		"""Function received the test to work on it to find the regexp

	"""Function received the test to work on it to find the regexp
	"""Function received the text to work on it to find the regexp

	The part or full text to be modified
	The partial or complete text to be modified.

[New Module] zos_replace #2044

Are you sure you want to change the base?

[New Module] zos_replace #2044

Uh oh!

Conversation

AndreMarcel99 commented Apr 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

SUMMARY

ISSUE TYPE

COMPONENT NAME

ADDITIONAL INFORMATION

Uh oh!

richp405 left a comment

Choose a reason for hiding this comment

Uh oh!

fernandofloresg left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AndreMarcel99 commented May 16, 2025

Uh oh!

rexemin left a comment

Choose a reason for hiding this comment

Uh oh!

surendrababuravella left a comment

Choose a reason for hiding this comment

Uh oh!

fernandofloresg left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

fernandofloresg left a comment

Choose a reason for hiding this comment

Uh oh!

AndreMarcel99 commented Jun 2, 2025

Uh oh!

ddimatos left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fernandofloresg Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

AndreMarcel99 commented Apr 9, 2025 •

edited

Loading

fernandofloresg Jun 4, 2025 •

edited

Loading